fbpx
Connect with us

Tech

Researchers are working toward more transparent language models

Published

on

Researchers are working toward more transparent language models

Image Credit: raindrop74 / Shutterstock

Hear from CIOs, CTOs, and other C-level and senior execs on data and AI strategies at the Future of Work Summit this January 12, 2022. Learn more


The most sophisticated AI language models, like OpenAI’s GPT-3, can perform tasks from generating code to drafting marketing copy. But many of the underlying mechanisms remain opaque, making these models prone to unpredictable — and sometimes toxic — behavior. As recent research has shown, even careful calibration can’t always prevent language models from making sexist associations or endorsing conspiracies.

Newly proposed explainability techniques promise to make language models more transparent than before. While they aren’t silver bullets, they could be the building blocks for less problematic models — or at the very least models that can explain their reasoning.

Citing sources

A language model learns the probability of how often a word occurs based on sets of example text. Simpler models look at the context of a short sequence of words, whereas larger models work at the level of phrases, sentences, or paragraphs. Most commonly, language models deal with words — sometimes referred to as tokens.

Indeed, the largest language models learn to write humanlike text by internalizing billions of examples from the public web. Drawing on sources like ebooks, Wikipedia, and social media platforms like Reddit, they make inferences in near-real-time.

Many studies demonstrate the shortcomings of this training approach. Even GPT-3 struggles with nuanced topics like morality, history, and law; language models writ large have been shown to exhibit prejudices along race, ethnic, religious, and gender lines. Moreover, language models don’t understand language the way humans do. Because they typically pick up on only a few key words in a sentence, they can’t tell when words in a sentence are jumbled up — even when the new order changes the meaning.

A recent paper coauthored by researchers at Google outlines a potential, partial solution: a framework called Attributable to Identified Sources. It’s designed to evaluate the sources (e.g., Reddit and Wikipedia) from which a language model might pull when, for example, answering a particular question. The researchers say that the framework can be used to assess whether statements from a model were derived from a specific source. With it, users can figure out to which source the model is attributing its statements, showing evidence for its claims.

“With recent improvements in natural language generation … models for various applications, it has become imperative to have the means to identify and evaluate whether [model] output is only sharing verifiable information about the external world,” the researcher wrote in a paper. “[Our framework] could serve as a common framework for measuring whether model-generated statements are supported by underlying sources.”

The coauthors of another study take a different tack to language model explainability. They propose leveraging “prototype” models — Proto-Trex — incorporated into a language model’s architecture that can explain the reasoning process behind the model’s decisions. While the interpretability comes with a trade-off in accuracy, the researchers say that the results are “promising” in providing helpful explanations that shed light on language models’ decision-making.

In the absence of a prototype model, researchers at École Polytechnique Fédérale de Lausanne (EPFL) generated “knowledge graph” extracts to compare variations of language models. (A knowledge graph represents a network objects, events, situations, or concepts and illustrates the relationship between them.) The framework can identify the strengths of each model, the researchers claim, allowing users to compare models, diagnose their strengths and weaknesses, and identify new datasets to improve their performance.

“These generated knowledge graphs are a large step towards addressing the research questions: How well does my language model perform in comparison to another (using metrics other than accuracy)? What are the linguistic strengths of my language model? What kind of data should I train my model on to improve it further?” the researchers wrote. “Our pipeline aims to become a diagnostic benchmark for language models, providing an alternate approach for AI practitioners to identify language model strengths and weaknesses during the model training process itself.”

Limitations to interpretability

Explainability in large language models is by no means a solved problem. As one study found, there’s an “interpretability illusion” that arises when analyzing a popular architecture of language model called bidirectional encoder representations from transformers (BERT). Individual components of the model may incorrectly appear to represent a single, simple concept, when in fact that they’re representing something far more complex.

There’s another, more existential pitfall in model explainability: over-trust. A 2018 Microsoft study found that transparent models can make it harder for non-experts to detect and correct a model’s mistakes. More recent work suggests that interpretability tools like Google’s Language Interpretability Tool, particularly those that give an overview of a model via data plots and charts, can lead to incorrect assumptions about the dataset and models, even when the output is manipulated to show explanations that make no sense.

It’s what’s known as the automation bias — the propensity for people to favor suggestions from automated decision-making systems. Combating it isn’t easy, but researchers like Georgia Institute of Technology’s Upol Ehsan believe that explanations given by “glassbox” AI systems, if customized to people’s level of expertise, would go a long way.

“The goal of human-centered explainable AI is not just to make the user agree to what the AI is saying. It is also to provoke reflection,” Ehsan said, speaking to MIT Tech Review.

VentureBeat

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative technology and transact.

Our site delivers essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the subjects of interest to you
  • our newsletters
  • gated thought-leader content and discounted access to our prized events, such as Transform 2021: Learn More
  • networking features, and more

Become a member

Go to Source

Click to comment

Leave a Reply

Tech

Intel tips plans for a Bitcoin mining ‘Bonanza’ chip

Published

on

Intel tips plans for a Bitcoin mining ‘Bonanza’ chip

Metamorworks/Nature/Getty Images

Note: When you purchase something after clicking links in our articles, we may earn a small commission. Read our affiliate link policy for more details.

  • CPUs and Processors
  • Cryptocurrency

As PCWorld’s senior editor, Mark focuses on Microsoft news and chip technology, among other beats. He has formerly written for PCMag, BYTE, Slashdot, eWEEK, and ReadWrite.

Go to Source

Continue Reading

Tech

Xbox Game Pass adds Death’s Door, Danganronpa in second half of January

Published

on

Xbox Game Pass adds Death’s Door, Danganronpa in second half of January

Join gaming leaders, alongside GamesBeat and Facebook Gaming, for their 2nd Annual GamesBeat & Facebook Gaming Summit | GamesBeat: Into the Metaverse 2 this upcoming January 25-27, 2022. Learn more about the event. 


Xbox announced today the second batch of games coming to Game Pass in January. It’s not as exciting as the news that Microsoft is buying Activision Blizzard, but hopefully it’s an indicator of more Game Pass additions to come. January’s games include Death’s Door, Danganronpa: Trigger Happy Havoc, and Pupperazi. Subscribers will also get The Hitman Trilogy and Rainbow Six Extraction this month.

Danganronpa: Trigger Happy Havoc and Nobody Saves the World launched today for Game Pass subscribers. The former is a murder mystery visual novel game, while the latter is a cartoony RPG. Nobody Saves the World is a day-one launch on Game Pass.

Most of the other games launching on Game Pass this month will launch on January 20. These include Death’s Door, the indie action-adventure title, and dog photography game Pupperazzi. Another January 20 launch is Windjammers 2, a disc-throwing game with hand-drawn graphics. Microsoft previously revealed that Rainbow Six Extraction would be a day-one release on Game Pass. The Hitman Trilogy will also launch on Game Pass. Yes, both of those will launch on January 20.

The final Game Pass launch of the month is Taiko no Tatsujin: The Drum Master. This drum-based rhythm game is also the only game in this batch that’s not available for all of Game Pass’s platforms. It’ll be available for console and cloud but not PC.

Event

The 2nd Annual GamesBeat and Facebook Gaming Summit and GamesBeat: Into the Metaverse 2

January 25 – 27, 2022


Learn More

Game Pass is having a banner day today. Microsoft revealed in the announcement of its Activision-Blizzard that Game Pass had surpassed 25 million subscribers. It also added that it would bring “as many Activision Blizzard games as we can within Xbox Game Pass and PC Game Pass, both new titles and games from Activision Blizzard’s incredible catalog.”

The games leaving Game Pass to make way for this batch are Cyber Shadow, Nowhere Prophet, Prison Architect, and Xeno Crisis. They’ll leave the service on January 31.

GamesBeat

GamesBeat’s creed when covering the game industry is “where passion meets business.” What does this mean? We want to tell you how the news matters to you — not just as a decision-maker at a game studio, but also as a fan of games. Whether you read our articles, listen to our podcasts, or watch our videos, GamesBeat will help you learn about the industry and enjoy engaging with it.

How will you do that? Membership includes access to:

  • Newsletters, such as DeanBeat
  • The wonderful, educational, and fun speakers at our events
  • Networking opportunities
  • Special members-only interviews, chats, and “open office” events with GamesBeat staff
  • Chatting with community members, GamesBeat staff, and other guests in our Discord
  • And maybe even a fun prize or two
  • Introductions to like-minded parties

Become a member

Go to Source

Continue Reading

Tech

Samsung’s newest mobile processor has an AMD GPU with ray tracing

Published

on

Samsung’s newest mobile processor has an AMD GPU with ray tracing

Samsung Galaxy Z Fold3

Alex Walker-Todd/IDG

Note: When you purchase something after clicking links in our articles, we may earn a small commission. Read our affiliate link policy for more details.

  • Smartphones

Michael is a former graphic designer who’s been building and tweaking desktop computers for longer than he cares to admit. His interests include folk music, football, science fiction, and salsa verde, in no particular order.

Go to Source

Continue Reading
Home | Latest News | Tech | Researchers are working toward more transparent language models
a

Market

Trending