The AI Community On Kbin

Stability AI Chief Executive Officer Emad Mostaque has resigned from the British artificial intelligence startup — a move that follows quarrels with investors and waves of senior staff departures.

AI in manufacturing sees the convergence of smart computing and algorithms with tasks that dictate the intricacies of the work.

AI is poised to revolutionize our understanding of animal communication

Training AI models like GPT-3 on "A is B" statements fails to let them deduce "B is A" without further training, exhibiting a flaw in generalization. (https://arxiv.org/pdf/2309.12288v1.pdf)

Ongoing Scaling Trends

• 10 years of remarkable increases in model scale and performance.

• Expects next few years will make today's AI "pale in comparison."

• Follows known patterns, not theoretical limits.

No Foreseeable Limits

• Skeptical of claims certain tasks are beyond large language models.

• Fine-tuning and training adjustments can unlock new capabilities.

• At least 3-4 more years of exponential growth expected.

Long-Term Uncertainty

• Can't precisely predict post-4-year trajectory.

• But no evidence yet of diminishing returns limiting progress.

• Rapid innovation makes it hard to forecast.

TL;DR: Anthropic's CEO sees no impediments to AI systems continuing to rapidly scale up for at least the next several years, predicting ongoing exponential advances.

Paper: https://arxiv.org/abs/2309.07124

Abstract:

Large language models (LLMs) often demonstrate inconsistencies with human preferences. Previous research gathered human preference data and then aligned the pre-trained models using reinforcement learning or instruction tuning, the so-called finetuning step. In contrast, aligning frozen LLMs without any extra data is more appealing. This work explores the potential of the latter setting. We discover that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting. We introduce a novel inference method, Rewindable Auto-regressive INference (RAIN), that allows pre-trained LLMs to evaluate their own generation and use the evaluation results to guide backward rewind and forward generation for AI safety. Notably, RAIN operates without the need of extra data for model alignment and abstains from any training, gradient computation, or parameter updates; during the self-evaluation phase, the model receives guidance on which human preference to align with through a fixed-template prompt, eliminating the need to modify the initial prompt. Experimental results evaluated by GPT-4 and humans demonstrate the effectiveness of RAIN: on the HH dataset, RAIN improves the harmlessness rate of LLaMA 30B over vanilla inference from 82% to 97%, while maintaining the helpfulness rate. Under the leading adversarial attack llm-attacks on Vicuna 33B, RAIN establishes a new defense baseline by reducing the attack success rate from 94% to 19%.

Source: https://old.reddit.com/r/singularity/comments/16qdm0s/rain_your_language_models_can_align_themselves/

https://arxiv.org/abs/2309.11495

Abstract

Generation of plausible yet incorrect factual information, termed hallucination, is an unsolved issue in large language models. We study the ability of language models to deliberate on the responses they give in order to correct their mistakes. We develop the Chain-of-Verification (CoVe) method whereby the model first (i) drafts an initial response; then (ii) plans verification questions to fact-check its draft; (iii) answers those questions independently so the answers are not biased by other responses; and (iv) generates its final verified response. In experiments, we show CoVe decreases hallucinations across a variety of tasks, from list-based questions from Wikidata, closed book MultiSpanQA and longform text

https://i.imgur.com/TDXcdMI.jpeg

https://i.imgur.com/XfRVxJT.jpeg

Conclusion

We introduced Chain-of-Verification (CoVe), an approach to reduce hallucinations in a large language model by deliberating on its own responses and self-correcting them. In particular, we showed that models are able to answer verification questions with higher accuracy than when answering the original query by breaking down the verification into a set of simpler questions. Secondly, when answering the set of verification questions, we showed that controlling the attention of the model so that it cannot attend to its previous answers (factored CoVe) helps alleviate copying the same hallucinations. Overall, our method provides substantial performance gains over the original language model response just by asking the same model to deliberate on (verify) its answer. An obvious extension to our work is to equip CoVe with tool-use, e.g., to use retrieval augmentation in the verification execution step which would likely bring further gains.

Source: https://old.reddit.com/r/singularity/comments/16qcdsz/research_paper_meta_chainofverification_reduces/

How Daron Acemoglu, one of the world's most respected experts on the economic effects of technology, learned to start worrying and fear AI.

Will everything you learned in college be replaced by ChatGPT? The CEO of job site Indeed says it's not out of the question.

Subscribe for good luck - Most Popular videos - https://www.youtube.com/watch?v=72qmtr41iWU&list=UULP3MF3KCtKQkCudSvNpk3BOg#liminalspace , #ai , #aiart _Arti...

cross-posted from: https://lemmy.ml/post/5325676

The past few months have launched generative AI models into the public eye, and everyone seems to have a take on it. Generative AI models such as large language models (LLMs) and AI art generators consume vast amounts of aggregated content, determine similarities between that content, and, when prompted, produce statistically likely, plausible-seeming output.

The current state of generative AI is environmentally disastrous and built on the backbone of labor exploitation, particularly in the global south. Large language models' disregard for the truth is, at this point, well-documented.

Technology is not neutral. Leveraging and normalizing generative AI is not a neutral act.

Subscribe for good luck - Most Popular videos - https://www.youtube.com/watch?v=72qmtr41iWU&list=UULP3MF3KCtKQkCudSvNpk3BOg#aimusic , #ai , #aiart _Artificia...

The latest updates to Google’s generative AI chat bot lets it dig through your personal email, documents, and more—so you can get things done faster.

Many American CEOs say they're worried about their workplace's lack of AI skills, a new survey of C-suite executives and workers found. Here's why.

The pursuit of the most advanced AI—human-like artificial general intelligence—has prompted concerns among experts about potential dangers if it runs amok.

63 percent of Americans want regulation to prevent artificial general intelligence, or AGI, which OpenAI aims to build.

Subscribe for good luck - Most Popular videos - https://www.youtube.com/watch?v=72qmtr41iWU&list=UULP3MF3KCtKQkCudSvNpk3BOg#chatgpt , #ai , #art_Artificial I...

These clever AI tools can have a big impact by using elaborate models to tackle demanding tasks. The nine programs presented here have something in common besides AI: they are freely available.

Stephen Fry said he was "shocked" when AI cloned his voice because it could "have me read anything from a call to storm Parliament to hard porn."

I think we have an answer on whether AIs will reshape work....

Subscribe for good luck - Most Popular videos - https://www.youtube.com/watch?v=72qmtr41iWU&list=UULP3MF3KCtKQkCudSvNpk3BOg#superman , #aiart , #joker _Arti...

Ethical AI requires a deep understanding of what there is, what we want, what we think we know, and how intelligence unfolds.