Meta tags:
description=We are Forethought, a research organization focussed on how to navigate the transition to superintelligent AI systems.;

Headings (most frequently used words):

ai, our, podcast, how, we, for, intelligence, explosion, to, research, about, should, navigate, explosive, progress, preparing, the, enabled, coups, small, group, could, use, seize, power, better, futures, will, automation, cause, software, tools, existential, security, focus, on, these, questions, listen, subscribe, careers, donate, contact, newsletter, search, tom, davidson, william, macaskill, mia, taylor, featured, people, forecast,

Text of the page (most frequently used words):
and (39), #research (19), the (18), for (17), systems (16), that (16), could (16), more (12), new (11), can (10), are (10), how (9), should (9), our (8), which (8), challenges (8), read (8), these (8), about (7), with (7), tools (7), newsletter (6), address (6), what (6), enabled (6), development (6), progress (6), will (6), human (6), now (5), podcast (5), subscribe (5), future (5), capable (5), even (5), improve (5), 2025 (5), advanced (5), coup (5), superhuman (5), email (4), might (4), help (4), technological (4), small (4), intelligence (4), explosion (4), just (4), companies (4), accelerate (4), projects (4), automated (4), search (3), forethought (3), your (3), applications (3), avoid (3), coups (3), soon (3), than (3), rapid (3), quickly (3), nonprofit (3), navigate (3), bio (3), fellow (3), senior (3), william (3), macaskill (3), tom (3), davidson (3), preparing (3), futures (3), power (3), technologies (3), this (3), there (3), already (3), work (3), they (3), march (3), from (3), feedback (3), loop (3), increase (3), military (3), government (3), persuasion (3), use (3), developments (3), succession (3), contact (2), donate (2), careers (2), subscribing (2), you (2), agree (2), substack (2), unsubscribe (2), anytime (2), archives (2), terms (2), service (2), enter (2), subscription (2), listen (2), great (2), agi (2), like (2), right (2), humans (2), aligned (2), face (2), autocracies (2), races (2), grab (2), offworld (2), resources (2), digital (2), well (2), opportunities (2), dramatically (2), quality (2), life (2), collective (2), decision (2), making (2), world (2), researchers (2), stay (2), latest (2), meet (2), transformative (2), governments (2), questions (2), data (2), key (2), include (2), risk (2), not (2), but (2), security (2), today (2), training (2), significantly (2), experiments (2), argue (2), such (2), runaway (2), need (2), additional (2), efforts (2), algorithms (2), improving (2), approach (2), series (2), better (2), group (2), leaders (2), fully (2), them (2), gain (2), weapons (2), frontier (2), multiple (2), loyal (2), researching (2), prepare (2), happens (2), miracle (2), drugs (2), bioweapons (2), militaries (2), prediction (2), many (2), arise (2), quick (2), accelerating (2), innovation (2), may (2), become (2), scientists (2), articles, designed, privacy, policy, non, profit, company, 16156175, post, look, most, other, thinking, much, were, conundrums, treat, minds, leading, focused, transition, superintelligent, favorite, platform, never, miss, episode, follow, links, below, view, all, episodes, expert, guests, discuss, plus, date, narrations, forecast, full, team, mia, taylor, people, speed, roughly, weekly, reach, really, good, rather, avoiding, catastrophe, beneficial, accelerated, extreme, concentrations, unlock, come, first, focus, priority, area, lots, get, started, plan, abundant, cognition, important, investing, task, relevant, lowering, adoption, barriers, securing, compute, while, background, limits, potential, gains, speedups, decisive, judgement, diverse, groups, identify, towards, shared, goals, specific, targeted, coordination, epistemic, humanity, prepared, driven, anticipate, together, available, time, lizka, vaintrob, owen, cotton, barratt, existential, increasingly, using, write, code, analyze, papers, generate, potentially, automating, entire, cycle, formulating, designing, implementing, testing, refining, trigger, develop, itself, speeds, resulting, extremely, fast, without, computer, chips, empirical, evidence, rate, suggests, positive, overcome, diminishing, returns, continued, evaluate, two, bottlenecks, scratch, takes, months, often, requires, computationally, expensive, however, find, possible, workarounds, enable, nonetheless, daniel, eth, automation, cause, software, one, near, term, catastrophes, extinction, essay, explores, different, complementary, where, survive, achieve, truly, suppose, want, august, broadly, create, serious, threat, staged, very, single, person, occur, established, democracies, sufficiently, introduce, three, novel, dynamics, firstly, replace, personnel, eliminating, supporters, secondly, deliberately, build, example, autonomous, robots, pass, tests, later, execute, when, deployed, settings, thirdly, officials, within, capabilities, strategic, planning, cyber, offense, their, until, stage, risks, design, enforce, rules, against, misuse, audit, secret, loyalties, share, stakeholders, establish, principles, oversight, procure, critical, independent, providers, exclusive, access, secretly, singularly, april, lukas, finnveden, rose, hadshar, seize, cannot, always, delegated, suggest, things, meaningfully, prospects, preparedness, therefore, ensuring, disorienting, range, would, bring, mass, destruction, beings, worthy, moral, consideration, drive, century, over, few, years, during, period, political, raise, consequential, hard, reverse, decisions, call, grand, fin, moorhouse, featured, explosive,

Text of the page (random words):
forethought research about careers donate contact podcast newsletter how should we navigate explosive ai progress ai is already accelerating innovation and may soon become as capable as human scientists if that happens many new technologies could arise in quick succession new miracle drugs and new bioweapons automated companies and automated militaries superhuman prediction and superhuman persuasion we are a nonprofit researching what can we do now to prepare ai is already accelerating innovation and may soon become as capable as human scientists if that happens many new technologies could arise in quick succession new miracle drugs and new bioweapons automated companies and automated militaries superhuman prediction and superhuman persuasion we are a nonprofit researching what can we do now to prepare featured research preparing for the intelligence explosion william macaskill fin moorhouse march 2025 ai that can accelerate research could drive a century of technological progress over just a few years during such a period new technological or political developments will raise consequential and hard to reverse decisions in rapid succession we call these developments grand challenges these challenges include new weapons of mass destruction ai enabled autocracies races to grab offworld resources and digital beings worthy of moral consideration as well as opportunities to dramatically improve quality of life and collective decision making we argue that these challenges cannot always be delegated to future ai systems and suggest things we can do today to meaningfully improve our prospects agi preparedness is therefore not just about ensuring that advanced ai systems are aligned we should be preparing now for the disorienting range of developments an intelligence explosion would bring read more ai enabled coups how a small group could use ai to seize power tom davidson lukas finnveden rose hadshar april 2025 the development of ai that is more broadly capable than humans will create a new and serious threat ai enabled coups an ai enabled coup could be staged by a very small group or just a single person and could occur even in established democracies sufficiently advanced ai will introduce three novel dynamics that significantly increase coup risk firstly military and government leaders could fully replace human personnel with ai systems that are singularly loyal to them eliminating the need to gain human supporters for a coup secondly leaders of ai projects could deliberately build ai systems that are secretly loyal to them for example fully autonomous military robots that pass security tests but later execute a coup when deployed in military settings thirdly senior officials within ai projects or the government could gain exclusive access to superhuman capabilities in weapons development strategic planning persuasion and cyber offense and use these to increase their power until they can stage a coup to address these risks ai projects should design and enforce rules against ai misuse audit systems for secret loyalties and share frontier ai systems with multiple stakeholders governments should establish principles for government use of advanced ai increase oversight of frontier ai projects and procure ai for critical systems from multiple independent providers read more better futures series william macaskill august 2025 suppose we want the future to go better what should we do one approach is to avoid near term catastrophes like human extinction this essay series explores a different complementary approach improving on futures where we survive to achieve a truly great future read more will ai r d automation cause a software intelligence explosion daniel eth tom davidson march 2025 ai companies are increasingly using ai systems to accelerate ai research and development today s ai systems help researchers write code analyze research papers and generate training data future systems could be significantly more capable potentially automating the entire ai development cycle from formulating research questions and designing experiments to implementing testing and refining new ai systems we argue that such systems could trigger a runaway feedback loop in which they quickly develop more advanced ai which itself speeds up the development of even more advanced ai resulting in extremely fast ai progress even without the need for additional computer chips empirical evidence on the rate at which ai research efforts improve ai algorithms suggests that this positive feedback loop could overcome diminishing returns to continued ai research efforts we evaluate two additional bottlenecks to rapid progress training ai systems from scratch takes months and improving ai algorithms often requires computationally expensive experiments however we find that there are possible workarounds that could enable a runaway feedback loop nonetheless read more ai tools for existential security lizka vaintrob owen cotton barratt march 2025 humanity is not prepared for the ai driven challenges we face but the right ai tools could help us to anticipate and work together to meet these challenges if they re available in time we can and should accelerate these tools key applications include 1 epistemic tools which improve human judgement 2 coordination tools which help diverse groups work identify and work towards shared goals 3 risk targeted tools to address specific challenges we can accelerate important tools by investing in task relevant data lowering adoption barriers and securing compute for key r d while background ai progress limits potential gains even small speedups could be decisive this is a priority area there is lots to do already and there will quickly be more we should get started and we should plan for a world with abundant cognition read more more research we focus on these questions what are the new technologies and challenges that ai could unlock which will come first what can companies and governments do to avoid extreme power concentrations which beneficial applications of ai should be accelerated how can we do that how do we reach really good futures rather than just avoiding catastrophe email address for newsletter subscription enter your email address to subscribe to our newsletter about transformative ai research subscribe stay up to speed with the latest research on preparing for transformative ai roughly weekly by subscribing you agree to substack s terms of service unsubscribe anytime archives our people tom davidson senior research fellow read bio william macaskill senior research fellow read bio mia taylor research fellow read bio meet the full team our podcast forecast listen to our researchers and expert guests discuss how to navigate the intelligence explosion plus stay up to date with narrations of our latest research view all episodes subscribe on your favorite platform and never miss an episode follow the links below or search for forethought podcast about we are a small research nonprofit focused on how to navigate the transition to a world with superintelligent ai systems ai systems might soon be much more capable than humans quickly leading to rapid technological progress even if ai systems were aligned we might face ai enabled autocracies races to grab offworld resources and conundrums about how to treat digital minds as well as opportunities to dramatically improve quality of life and collective decision making right now we are thinking about how ai and technological development might go how to avoid ai enabled coups which applications of ai can most help with other challenges what a great post agi future might look like research listen to our podcast email address for newsletter subscription enter your email address to subscribe to our newsletter subscribe by subscribing you agree to substack s terms of service unsubscribe anytime archives research about careers donate contact podcast newsletter forethought research is a non profit company no 16156175 privacy policy designed by and now search search for articles