56
If AI is making the Turing test obsolete, what might be better?
(arstechnica.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 15 Dec 2023
56 points (100.0% liked)
Technology
37801 readers
261 users here now
A nice place to discuss rumors, happenings, innovations, and challenges in the technology sphere. We also welcome discussions on the intersections of technology and society. If it’s technological news or discussion of technology, it probably belongs here.
Remember the overriding ethos on Beehaw: Be(e) Nice. Each user you encounter here is a person, and should be treated with kindness (even if they’re wrong, or use a Linux distro you don’t like). Personal attacks will not be tolerated.
Subcommunities on Beehaw:
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 2 years ago
MODERATORS
It's not making Turing test obsolete. It was obvious from day 1 that Turing test is not an intelligence test. You could simply create a sufficiently big dictionary of "if human says X respond with Y" and it would fool any person that its talking with a human with 0 intelligence behind it. Turing test was always about checking how good a program is at chatting. If you want to test something else you have to come up with other test. If you want to test chat bots you will still use Turing test.
Sounds to me like that sufficiently large dictionary would be intelligent. Like, a dictionary that can produce the correct response to every thing said sounds like a system that can produce the correct response to any thing said. Like, that system could advise you on your career or invent machines or whatever.
So would a book could be considered intelligent if it was large enough to contain the answer to any possible question? Or maybe the search tool that simply matches your input to the output the book provides, would that be intelligence?
To me, something can't be considered intelligent if it lacks the ability to learn.
Definitely. If you have a search tool that maps situational data to the perfect response, like it works out well every time, your search tool is intelligent. Period.
So given a large enough text file, the ctrl+f search box in Notepad would constitute as being intelligent in your opinion?
No, a dictionary is not intelligent. A dictionary simply matches one text to another. A HashMap is not intelligent. But it can fool a human that it is.
Yes, but you could argue that human brain is a large pattern matcher with a dictionary. What separates human intelligence from machine intelligence?
The question is not if something is a patter matcher or not. The question is how this matching is done. There are ways we consider intelligent and ways that are not. Human brain is generally considered intelligent, some algorithms using heuristics or machine learning would be considered artificial intelligence, a hash map matching string A to string B is not in any way intelligent. But all this methods can produce the same results so it's impossible to determine if something is intelligent or not without looking inside the black box.
Any hash map you or I have ever seen is not very intelligent, possibly not at all. But the infinitely large hash map we’re talking of is different. It can handle any possible situation it encounters. That’s part of its definition.
Our hashmaps — the finite hashmaps we use to store shipping addresses and candy crush preferences — would be torn to shreds in the real world. But not this infinite hashmap that maps all possible inputs to all possible outputs. It’s a one-layer network but it’s really wide. It’s as wide as the universe of possibility, at least.
No, infinite hash map is still not intelligent, not even by the standards used in computer science. It's not a one-layer network, it's not a network at all. To talk about network nodes form layer 1 would have to connect to multiple nodes in layer 2. The signal would have to be processed somehow. Extremely big one layer neural network could be intelligent for all we know. In theory some consciousness could emerge from sufficiently complex system like that. In a hash map there's no processing though, not matter how big it is. You simply take element A and return element B mapped to it. The operation is always the same. Making this map bigger does not add complexity, knowledge or alter how it's processing inputs. Big hash map is just like a small hash map, only bigger.
Yes, but we have no strict or clear s ientific definition of what makes humans intelligent or what intelligence even is.
Humans are intelligent and machines are not "just because"
Yes, we don't have a universal definition of intelligence but we in general everyone would agree that knowledge is not intelligence. Simply storing information does not make anything intelligent. Book is not intelligent, Wikipedia is not intelligent, hash map is not intelligent.
Yes, but we also have to draw a line somewhere. You could just as well turn any non-random based computer program into a huge hashtable, yet the intelligence arises from somewhere. There is no magic to human intelligence, unless you start believing in the soul or something.
Yes, that's the whole point. You can turn substitute computer program by a hash map and the results would be the same but everyone in general agree that a hash map is not intelligent. Defining exactly why it's not intelligent is tricky though. It comes down to some very basic concepts that we understand intuitively but are very hard precisely define like what it means to 'know' something or to 'understand' something. One famous example is a very good dictionary: let's say some guy has a very good Chinese dictionary. A Chinese speaking person can write question down and give it to this guy. He will look up every symbol in the question, translate it to English, respond and translate the response back to Chinese using the same dictionary. Does he 'speak' Chinese? He can communicate in Chinese but obviously he does not speak it. Does he 'understand' Chinese? Again, not really, he can just look up symbols in a dictionary. Specifying the exact reason why we would not say that he can 'speak' Chinese is difficult thought. It's the same with intelligence. We intuitively understand why a book is not intelligent but to say exactly why is tricky.
Yes but you are missing my point. We have no way of measuring if a human is intelligent. The whole intelligence might just as well be an illusion.
Yet language and abstraction are the core of intelligence. You cannot have intelligence without 2 way communication, and if anything, your brain contains exactly that dictionary you describe. Ask any verbal autistic person, and 90% of their conversations are scripted to a fault. However, there's another component to intelligence that the Turing Test just scrapes against. I'm not philosophical enough to identify it, but it seems like the turing test is looking for lightning by listening for rumbling that might mean thunder.
If you want to get philosophical the truth it we don't know what intelligence is and there's no way to identify it in a black box. We may say that something behaves intelligently or not but we will never be able say if it's really intelligent. Turing test check if a program is able to chat intelligently. We can come up with a test for solving math intelligently or driving car intelligently but we will never have a test for what most people understand as intelligence.
This is what it comes down to. Until we agree on a testable definition of "intelligence" (or sentience, sapience, consciousness or just about any descriptor of human thought), it's not really science. Even in nature, what we might consider intelligence manifests in different organisms in different ways.
We could assume that when people say intelligence they mean human-like intelligence. That might be narrow enough to test, but you'd probably still end up failing some humans and passing some trained models
It's not that it's not science. Different sciences simply define intelligence in different ways. In psychology it's mostly the ability to solve problems by reasoning so 'human like' intelligence. They don't care that computers can solve the same problems without reasoning (by brute force for example) because they don't study computers. In computer science it's more fuzzy but pretty much boils down to algorithms solving problems by using some sort of insights that are not simple step-by-step instructions. The problem is that with general AI we're trying to unify those definitions but when you do this both lose it's meanings.
You're right, it's very much context dependent, and I appreciate your incite on how this clash between psychology and computer science muddies the terms. As a CS guy myself who's just dipping my toes into NN's, I lean toward the psychology definition, where intelligence is measured by behavior.
In an artificial neural network, the algorithms that wrangle data and build a model aren't really what makes the decisions, they just build out the "body" (model, generator functions) and "environment" (data format), so to speak. If anything that code is more comparable to DNA than any state of mind. Training on data is where the knowledge comes from, and by making connections the model can "reason" a good answer with the correlations it found. Those processes are vague enough that I don't feel comfortable calling them algorithms, though. It's pretty divorced from cold, hard code.
Nah, I think a hash map is intelligent if and definitely if it maps all possible inputs. Then it’s intelligent. Don’t overestimate your own information content there, homo sapiens. You assume there’s no problem that your mind cannot solve, which is a weak assumption given the infinity of problems that awaits your species.
No, a hash map is not intelligent. There's no processing in the hash map. The input is not processed in any way, you directly use it to find the corresponding out put. Think about it this way: if you take a hash map with all possible inputs and print it out, will the paper be intelligent? You can still use this paper to map each input to an output, it holds all the same information the hash map did but obviously a mountain of paper is not intelligent. So you scan it back and store in a computer. Did it suddenly become intelligent now? Of course not, it's still just a static collection of information. Information is not intelligent.
So, ChatGPT?