this post was submitted on 26 Apr 2024
32 points (79.6% liked)

Futurology

1784 readers
76 users here now

founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] Umbrias@beehaw.org 8 points 6 months ago

Crazy how easy it is to poke holes in these ai studies.

We conducted a single evaluation for each AI model on August 1, 2023 of its SI performance using the Social Intelligence Scale (Sufyan, 1998). In each evaluation, we provided AI the same 64 standard SI scenarios.

So no repeated experiments, and using standard questions that are likely a part of the data set used to train the ai in the first place, with answers.

They didn't extend the test to anything useful. What a waste of time and money meant to hype ai.