ROME: Locating and Editing Factual Associations in GPT (Paper Explained & Author Interview)
#ai #language #knowledge
Large Language Models have the ability to store vast amounts of facts about the world. But little is known, how these models actually do this. This paper aims at discovering the mechanism and location of storage and recall of factual associations in GPT models, and then proposes a mechanism for the targeted editing of such facts, in form of a simple rank-one update to a single MLP layer. This has wide implications both for how we understand such models’ inner workings, and for our ability to gain greater control over such models in the future.
OUTLINE:
0:00 - Introduction
1:40 - What are the main questions in this subfield?
6:55 - How causal tracing reveals where facts are stored
18:40 - Clever experiments show the importance of MLPs
24:30 - How do MLPs store information?
29:10 - How to edit language model knowledge with precision?
36:45 - What does it mean to know something?
39:00 - Experimental Evaluation & the CounterFact benchmark
1 view
35
13
3 months ago 00:01:42 1
Chinese medical team performs groundbreaking remote surgery between Beijing and Rome
3 months ago 00:02:52 1
Empire - LEGIO XXI RAPAX & Römerstadt Carnuntum 🪔
3 months ago 00:00:27 1
This Italian Dancer Got a Full Body Adjustment w/ HUGE Neck Crack #Satisfying #Chiropractic
3 months ago 00:00:26 1
She Got To Experience My Favourite Technique & Couldn’t Believe It 💪 #chiropractic #shorts
3 months ago 00:00:43 1
Her Neck Popped So LOUD! 🤯 #shorts #chiropractic
4 months ago 00:10:57 1
Tempio della Fortuna Primigenia, Sanctuary of Fortuna Primigenia at Palestrina, manortiz
4 months ago 00:21:11 1
Don’t miss this bakery in Rome. Artisanal, authentic, minimalist, modern and so comfortable!
4 months ago 00:15:58 1
SEVERE SPINE DEFORMATION is FIXED with Chiropractic Cracking
4 months ago 01:11:10 1
TOP 50 / Beautiful Landmarks Around the World | Must-See Incredible Sights 60FPS 8K ULTRA HD
4 months ago 00:02:59 1
Alain Delon - Just Say I Love Him (by Timi Yuro) with lyrics
4 months ago 01:02:23 1
THE WORLD’S MOST SPECTACULAR VIEWS 60FPS 8K ULTRA HD
4 months ago 00:01:27 1
History of Latin Language - World History for Kids | Educational Videos by Mocomi
4 months ago 00:14:56 2
Shay Mitchell Talks New Travel Show “Thirst“ & Seeing Ashley Benson as a Mom
4 months ago 00:21:00 1
Italian Model Popped Like A Champagne Bottle! Huge Neck & Back Relief - ASMR Chiropractic Therapy
4 months ago 00:00:28 1
Italian Model Cracked SO LOUD! Which Was Your Favourite?🤌 #shorts
4 months ago 00:44:21 1
Palatine Hill, Archeological Wonder of Rome - Guided and Narrated - 🇮🇹 Italy [4K HDR] Walking Tour
4 months ago 00:12:11 1
The BEST Preserved Roman Colony in the World
5 months ago 00:04:22 10
MONUMENT OF MISANTHROPY - A Nice Beheading For MoM (OFFICIAL MUSIC VIDEO) Tech / Brutal Death Metal
5 months ago 00:58:39 10
The Italian Origins Of The Baroque Art Movement (Waldemar Januszczak) | Ep 1 | Perspective
5 months ago 00:13:22 1
Virtual Ancient Rome in 3D - Baths of Caracalla, 13 minute detailed video tour
5 months ago 00:23:33 3
A day at one of the worlds most Challenging airports, 4K Planespotting at Lukla Tenzing Hillary
5 months ago 00:02:42 1
David Garrett - CHELSEA GIRL
5 months ago 00:03:17 1
LACUNA COIL - Reckless (OFFICIAL VIDEO)
5 months ago 00:20:31 2
The Loudest POPS & CRACKS I’ve Ever Heard! Italian Model Receives FULL BODY Chiropractic Session