Is chatgpt proof that standard tests are bad measures of intelligence
kromem @ kromem @lemmy.world Posts 6Comments 1,655Joined 2 yr. ago
kromem @ kromem @lemmy.world
Posts
6
Comments
1,655
Joined
2 yr. ago
Actually, you can give chatbots a real IQ test, and the range of scores fall into roughly the same spread as how they rank on other measures, with the leading model scoring at 100:
https://www.maximumtruth.org/p/ais-ranked-by-iq-ai-passes-100-iq
IQ tests have the goal of measuring intelligence.
The range of LLM scores on IQ tests:
https://www.maximumtruth.org/p/ais-ranked-by-iq-ai-passes-100-iq
But maybe if you keep reading you'll find out it was actually leading into telling you about the fact that in 1998, The Undertaker threw Mankind off Hell In A Cell, and plummeted 16 ft through an announcer’s table.
Deleted
Permanently Deleted
Jump
One of the smarter ad analysts I knew likened ad spaces to ecosystems, where a bunch of companies come in with crap ads that aren't related to what people are actually in market for or are misleading, and act as polluters which turn people off from green pastures.
As an example, when mobile browsing was first getting off the ground CTR for mobile banner ads was 15%.
Reddit's metrics are about to go to shit.
Deleted
Permanently Deleted
Jump
I always kind of wonder how it would go if I ever actually answered the door to proselytizing folks.
They'd clearly be unprepared to answer about the late insertion of the fundraising reversal in Luke.
Or the ambiguous engendering of God.
Or the eroding of a matriarchal tradition in early Judaism.
Or that a lot of the stories were probably appropriated from elsewhere.
How long before they fold their arms and say "well I don't know about that" or "I'll have to ask my preacher"? Will I stay on their list for another round of discussion, or not?
To date the only folks I've interacted with around my neighborhood were Mormon missionaries, who are just so well mannered and youthfully naive I didn't have the heart to pull out a map that shows the distance between Manchester, NY and Jerusalem.
'Angry' proselytizers though I feel like I'd have at least a few words for their deaf ears to balance things out for the rest of the neighborhood having to hear their words in turn.
Copilot is the most gamer-y of all the chatbots I've seen.
Which is wild as it's just a snapshot of GPT-4 behind the scenes.
My best guess is that there's a context bias by its association in the System prompt to Microsoft which brings it closer to topics like Xbox and gaming than models that don't have that alignment cue.
Don't underestimate the franchise potential of The Whistleblower Bodyguard 4: Furious at Fast Food.
Your competitors take out contract hits against your whistleblower and you need to have bodyguards to protect them.
And then your head of security and the whistleblower fall in love until at the end of the movie the competitor assassin gets into the court waiting room and the head of security throws themselves into the ninja star's way and dies in the whistleblower's arms as the ultimate sacrifice is made for love and corporate profits.
I tear up just thinking about it.
No, it just implies that it was adaptive to look like a bird.
It could be for any number of reasons, including because aliens exist and years ago they were like "let's screw up all the plants in this area for generations" until the leader's kid saw one that kind of looked like little birds and threw themselves in front of it and said "wait, no, spare this one."
Yep. A bit like a 7 day publicly displayed tracker of days on a 28 day lunar calendar cycle.
Was "I am the God of your Father" an editorial attempt to distinguish the deity from the gods of Egypt, or from the god of a Mother?
There's some pretty odd details in that book, like in Isaac's supposed patriarchal blessing which discussed "the sons of your mother bow down to you" or it being the only place there's the male form of gebirah ("Great Lady") - a title first applied in the text to Isaac's mother whose name is based on the word for 'chief.' Who is supposedly later followed by a figure 'Deborah' ('bee') who is a leader of the people around the time we now know bees were being imported into Tel Rehov and regularly requeened to avoid genetic drift with local bee populations. Also weird that the events regarding a "land of milk and honey" supposedly take place in a land with no honey and only one discovered apiary.
That apiary gets burned down right around the time Asa allegedly deposed his grandmother the gebirah ("Great Lady").
Tangental, but who needs deepfakes when a bit of old school Photoshop is misleading enough.
Were there people that call themselves "Jews" attacking the Al Aqsa mosque and violently shooting Muslims one year ago? Do you think he is referring to that?
Considering the first article is from 2017 and the second from 2014, no I don't think either are referring to the Al Aswa mosque violence in 2023.
If they were, that would be really surprising.
Proved the atoms were arrangements of the four elements, not so much.
Wasn't the Epicurean position. Lucretius only surmises that there were likely a few handfuls of base forms of indivisible parts and then a multitude of their combinations. In fact, he rejects the elemental view.
And given we jumped the gun on naming 'atoms' after the word for indivisible, the closer philosophical parallel to modern concepts is quanta. And in that context, you even have Lucretius claiming that the behaviors of said indivisible parts must have a degree of indeterminate outcomes beyond following static physical laws for there to be free will (long before Bell's work relating the behavior of quanta to superderminism). He also surmised that light was made up of indivisible parts that were extremely light and moving very, very fast around 2,000 years before Einstein proved the discrete nature of light.
They were right about everything from survival to the fittest, contribution of traits from each parent, the quantization of light, and the indeterminate behaviors of quanta literally thousands of years before these things are proven.
It wasn't mere happenstance that they ended up being the most correct about the physical world of all the schools of philosophy in antiquity. They had a concrete methodology behind their success, and frankly it's a methodology that modernity would do well to have learned more from.
Yet I provide no supporting evidence, written and dated or not, thus I am no giant.
Much of Einstein's work we recognize as monumental were things that could not be proven in his time and were only validated decades later.
The Epicureans may not have had the scientific method available to them, but their focus on observation driven speculation was literally one of the factors that fed into its creation (see the Pulizer winning The Swerve).
In a July 21 lecture posted on the Davis Masjid YouTube channel, Muslim preacher Ammar Shahin spoke in English and Arabic about how all Muslims, not only Palestinians or Syrians, will be called upon to kill all the Jews on "the last day."
In a video translated by the Middle East Media Research Institute (MEMRI), Shahin also stressed that the Hadith (oral tradition of sayings attributed to the prophet of Islam) does not say where the final battle will take place. "If it is in Palestine," for example, "or another place," hinting at the possibility that such a battle could happen in the United States or Europe as well.
He also prayed that al-Aksa mosque be liberated from "the filth of the Jews."
From this link: https://m.jpost.com/Arab-Israeli-Conflict/California-Imam-calls-on-Allah-to-annihilate-Jews-500676
In a clip of the kids’ TV show “The Pioneers of Tomorrow,” broadcast on May 2 and uploaded Thursday by the MEMRI media watchdog, the host of the program, a young girl in a hijab, interviews two very young children, one of whom says she hopes to be a police officer like her uncle Ahmad.
The host asks what policemen do, and, after establishing that they catch criminals, adds that “they shoot Jews, right?” and stresses to her young guest that “you want to be like him.”
“I will shoot the Jews!” the little child says.
“All of them?” the host asks.
“Yes,” the girl says
“Good,” the host answers.
In a previous segment, a co-host of the show, an anthropomorphic bee character, talks on the phone with a child in the West Bank Jenin refugee camp, and encourages him, if Jews come into the camp, to “punch them” and “turn their faces into tomatoes in order to liberate Palestine.”
Yeah, nothing antisemitic here.
You pasted random Hasbara from internet that you didn't read and called criticism of israel anti-Semitic.
Citation needed, as it appears we're just making up straw men now. In this comment thread, I literally didn't even write the word 'antisemitism' outside of quoting your use of it.
I'm not sure why they are describing it as "a new paper" - this came out in May of 2023 (and as such notably only used GPT-3 and not GPT-4, which was where some of the biggest leaps to date have been documented).
For those interested in the debate on this, the rebuttal by Jason Wei (from the original emergent abilities paper and also the guy behind CoT prompting paper) is interesting: https://www.jasonwei.net/blog/common-arguments-regarding-emergent-abilities
In particular, I find his argument at the end compelling:
Another popular example of emergence which also underscores qualitative changes in the model is chain-of-thought prompting, for which performance is worse than answering directly for small models, but much better than answering directly for large models. Intuitively, this is because small models can’t produce extended chains of reasoning and end up confusing themselves, while larger models can reason in a more-reliable fashion.
If you follow the evolution of prompting in research lately, there's definitely a pattern of reliance on increased inherent capabilities.
Whether that's using analogy to solve similar problems (https://openreview.net/forum?id=AgDICX1h50) or self-determining the optimal strategy for a given problem (https://arxiv.org/abs/2402.03620), there's double digit performance gains in state of the art models by having them perform actions that less sophisticated models simply cannot achieve.
The compounding effects of competence alone mean that progress here isn't going to be a linear trajectory.
This isn't quite correct. There is the possibility of biasing the results with the training data, but models are performing well at things they haven't seen before.
For example, this guy took an IQ test, rewrote the visual questions as natural language questions, and gave the test to various LLMs:
https://www.maximumtruth.org/p/ais-ranked-by-iq-ai-passes-100-iq
These are questions with specific wording that the models won't have been trained on given he wrote them out fresh. Old models have IQ results that are very poor, but the SotA model right now scores a 100.
People who are engaging with the free version of ChatGPT and think "LLMs are dumb" is kind of like talking to a moron human and thinking "humans are dumb." Yes, the free version of ChatGPT has around a 60 IQ on that test, but it also doesn't represent the cream of the crop.