Slide
Topic / Claim / Title
Idea / Question / Problem / Detail
Citations / Relevant / Definitions
Related Orgs / Connection
Images
1
Intro /
AI and Ethereum Are “Two Sides of The Same Coin” → each has the unique potential to check and balance the excesses (+dangers +edge-cases) of the other, while both can supercharge Economic Progress and “resolve” Cybersecurity
- This overlap seems underexplored in policy discussions
Idea: We need new tools, formats, and interfaces (tailored to individuals using AI, made trustless with web3) for large-scale, time-efficient, and fun deliberation / decision-making / information-transfer / economic interaction as part of a coherent whole-society response to rapid AI capabilities progress → Ethereum, together with Quadratic Mechanisms and “Zero Knowledge” (ZK) math, constitutes an ideal platform for diverse such “Social Infrastructure” (consisting of Coordination Tools and Public Goods + associated positive externalities), and as a credibly neutral global arbiter / as a toolkit for designing previously infeasible or impossible incentive (+economic +organisational +logistical +political) systems
Idea: Ethereum is “AI Alignment Substrate”, and AI will make web3 easy to use for “non-technical” people → Highlights include
- Human / AI Data Provenance guarantees (see below)
- Scam Prevention
- Trustless chain-of-command and data responsibility / custody guarantees
- …
Problem: Hard to succinctly communicate about Ethereum (as coordination tech), ZK (for privacy-preserving digital id), and Quadratic Mechanisms (for better voting / resource allocation), although people like them in practice → Idea: We could use a fun and beautiful public-facing game/interface/tool/trial/“place” which familiarises people with these tools and perspectives (as a side-effect)
“Incentive Engineering” → Ethereum is Game-Changing Technology, Literally (Virgil Griffith)
- Ethereum as credibly-neutral “global arbiter”, as sandbox for better incentive-systems, and as only possible “AI Alignment Substrate” for coordinating around multipolar traps
Topics and Conversation @ intersection of web3 / AI (Kevin Owocki)
- Hard to communicate succinctly about Quadratic Mechanisms, but major potential for better processes:
- Quadratic Voting
- Quadratic Resource Alloc.
- “Techno-Georgism” – SALSA = Self Assessed Tax
- “Plurality” as moderation tool
Resource: Decentralized Society (DeSoc – E. Glen Weyl, Puja Ohlhaver, Vitalik Buterin)
- Uses of SBTs and ZK proofs for society
- We have incentive and means to create conditions for identifying top 1% of AI predictors
Resource: Greenpill (Kevin Owocki)
Regenerative Economics and Public Goods focus is crucial
ZK = “Zero Knowledge” cryptography
Privacy-Preserving math, Useful for Proof of:
- Humanity (need Govtscale. initiatives here)
- Arbitrary Credentials
- LLM Training Data / Model Versions, Parameters, “Supraprompts” → “ideal juries” of GPT-n agents → Opportunities for “Privacy-Preserving / Anonymous Surveillance”
Resource: Secret Societies, Network States, Burning Man, Zuzalu, And More: Thoughts On New Political Communities (Matt Prewitt)
- “... the work of building a democratic society is to disseminate enough information and education for the public to be able to discern itself. Information about the public’s problems must therefore be broadly available”
- “important implications for the interaction between community formation and cryptographic techniques including public blockchains, zero-knowledge proofs and other privacy enhancing technologies, and designated verifier signatures.”
AI Objectives Institute (AOI)
- Piloting New Tools: TalkToTheCity, LucidLens …
Collective Intelligence Project (CIP)
- Democratic Process Innovation Focus
- “Alignment Assemblies”
RadicalxChange (RxC)
- Quadratic Mechanisms for healthier societies, applicable to:
- DIgital Moderation
- Resource Allocation
- Future Taxation
- Next-Gen Voting Systems
- QF python implementations (Jmiller4)
Polis
- Discourse mapping tools and trials
Gitcoin (+ Schelling Point)
- Quadratic Fundraising and Public Goods Focus
- Gitcoin Passport
Plurality Network: Piloting New Tools, Collaboration Focus
Zuzalu: AI / Crypto Overlap
2
Risk Matrix
/
Attention Scarcity / Information Overwhelm Is a “meta-problem” for AI Safety and a subtle multiplier of various x-risks
Idea: Attention Scarcity / Information Overwhelm (and inadequate coordination technology / formats) constitutes a “meta-problem” underlying our difficulty in keeping up with AI progress (in an individual and societal way) which subtly multiplies other x-risk categories →
Idea: A “risk matrix” tool for collaboratively tracking (in a “heatmap” way) intersections (risks / opportunities) between major categories of [ existential risks / technologies / AI actors / incentive / policies ] – could help ensure we are covering all relevant intersections
- Categories can be expanded into subcategory layers (for added nuance), with “conceptual coverage” report generated, and subcategory values adding up to parent-category “heatmap” intensity values
Question: What are ideal categories for such a “Risk Matrix”? Relevant categories (with potential subcategories represented in 3D format) might include: →
Dunbar’s Number for x-risks (metaphor)
- Probably not possible for any individual to understand all the complex interactions across so many x-risk / relevant categories:
Technologies
AI
- LLM, Image, Video, Voice
Ethereum / ZK
- NFTs / “Soulbound Tokens” (SBTs)
- As Credentials (+reputation), As API / model-instance access-keys, As Collateral (+commitment to treaties), As Intellectual Property and Scientific Research Materials, As digital files embedded in / connected to objects / commodities
- ZK Proofs (see definition above)
Brain-Computer Interface (BCI), Synthetic Biology (Problem: Dual Use Risk [1]), Quantum Computers (Problem: broken ECDSA)
Other risks
Nuclear Weapons, Climate Change, …
Economic Forces
Ransomware / Impersonation / Scams, Job Automation → UBI Imperative, Abundant Fusion Energy (Potentially – similar endgame appears possible with “enhanced geothermal”)
Actors / Incentives
Govts (Domestic / International: Data Privacy, Economic Policy, Military), AI Labs, Individuals
Centre for the Study of Existential Risk (CSER)
- Studying & Tracking Existential Risks
AI Hotline
/
Incentive Problems
- “Arms Race” Dynamics
- Security Dilemma
/
Airgapped AI Facility
Idea: ZK whistleblowing application for AI lab employees (to partially ameliorate “arms race” dynamics) →
- Seed a prize-pool (for retroactive payment if reports/claims borne out), potentially funded by legislation
- Create a privacy-preserving (but “claimable” later using ZK) mechanism (via on-chain SBT Credentials) for AI lab employees to anonymously submit safety concerns / reports
- Push for legislation which allows Government / “Smart-Contract-Governed IAEA” investigation based on these claims
- Would be heartening to receive participation of AI labs, but their approval not necessary – SBT credential could be cryptographically linked to “proof of aicompany.com email address” process
- Preemptively disarm potential future tensions between general public / AI lab employees
Idea: Cryptographic killswitch for AI (how to manage)
Question: Should the public be able to elect x people who are allowed to spectate on the inside of AI labs (at some interval or consistently)? Would this improve incentives or simply be a pointless draconian measure? Would (or should) it be legal?
Question: Should major AI labs be “nationalized” into one (potentially publicly overseen, via smart-contract voting mechanisms) organization and/or “airgapped” facility? Could such a setup improve incentives? → Idea: Might be wise to “left curve it” and consider relative advantage of hardware solutions to AI containment where relevant / possible
SBT = “Soulbound Token” = Non-Transferrable NFT
- Example credentials which could be represented as SBTs:
- “Ability to use the full medical model”
- …
US-China relations, arms race dynamics, EU policy, and AI progress forecasting
Data
Data Provenance / Attestation
Idea: ZK + Ethereum can be used to build a privacy-preserving system of data provenance guarantees, for example:
- Prove a photo was taken by a human (via hardware in phones) but without revealing anything about the individual human
More ideas here, related to
- Trusting model weights / initializations for shared / public “Co-LLM” and “policy-suggestion” use-cases
- …
Question: If throughput / tx-costs no problem, what data should we focus on timestamping / attesting by default?
GPU Tracking /
Compute Governance /
Software By Specification /
Trusted Computation
We could likely have the best of both worlds when it comes to:
personal privacy vs. monitoring / legislating access to AI models and compute
Idea: Controversial measures, might be especially relevant for next few years (before algorithmic / OSS + ggml-type progress) – in long run might be better to token-gate (+ZK) access to dangerous “databases”, such as biorisk drug-discovery specialist model / training material
- If feasible and privacy preserving, might be one of the more powerful levers we have
- Could get creative with ZK applications, consider “conditional tracking”:
- Tracking GPUs: …
- Tracking LLM use: …
- ZK Firmware:
- Send alert when > x GPUs connect
- GPUs self-destruct if x happens
- Anonymous surveillance by “elected” gpt-n agents for societal harms
Question: Would society accept “privacy-preserving surveillance” if literacy (+trust) around possible ZK guarantees were high enough? Idea:More public trust in software might eventually stem from “software by specification” and where major public software is coded by gpt-n (with spec. potentially voted on) rather than created by specific humans → such software could involve automated formal proofs of correctness
Paper: Compute Monitoring / Tracking: What does it take to catch a Chinchilla? (Yonadav Shavit)
Interview + More Links: Compute Governance (Lennart Heim)
Resource: Compute Index (State of AI)
Article: How We Can Regulate AI (Avital Balwit)
Tool: ZCash Halo
- Trusted Computation
GPT: Yes, this seems broadly possible: using ZK to get best of both worlds for
personal privacy vs. monitoring / legislating access to AI models and compute
- chat snippet
Rethink Priorities
- Compute Tracking Research
- Quantitative Data about AI industry (example: compute index) and tracking of developments
US-China relations, arms race dynamics, EU policy, and AI progress forecasting
World Vote
/
The Worldcoin “Orb” and/or Apple Headset with OpticID / Iris Scanning arguably appears to be (at least part of) our best Anti-Sybil option
/
Anonymous Digital Voting and Timelocked Encryption
/
Universal Basic Income (UBI)
- see below
Ideas: re: Iris Scanning:
Many people will probably be suspicious of implementation no matter what, but there are better and worse ways to introduce such technologies to widespread use
- Orb/Apple OpticID should be audited in global way if it will be used on global population
- Critical software portions (for privacy guarantees) should be formally proven and 1-click-auditable
- Relevant Expertise: Psychologists / Historians / Anthropologists (re: Potential Worldwide Onboarding Policy)
- Onboarding could be incentivized / paid for by Govt, with privacy guarantees (via Software By Spec. and trusted “gpt juries” of auditor models) and massive bounties for any flaws or inconsistencies found in implementation
“Liquid Democracy”
- Key aspects: delegatable votes (and revokable delegations)
Radical Markets – Votes & UBI chapter
- Idea: “Vote Credits” and UBI could be somewhat interconnected to raise enough funds for UBI (this is controversial, for good reason, and should be debated / investigated)
Current ZK tools:
- 0xPARC (Zuzalu Passport)
- Functions: …
- Sismo (zkConnect)
- Functions: …
- ZCash Halo …
Apple (or other) products helping prepare society for public-key and anti-sybil infra adoption (?)
Resources: Anonymous Digital Voting
- Repo (stonecoldpat)
- Paper
- Aztec / Nouns “Private DAO Voting” initiative
- Tool: Timelocked Encryption
Resource: On biometric proof-of-personhood (Vitalik Buterin)
Democracy Earth (DE)
- Liquid Democracy and UBI
WorldCoin
- …
AGI Plans
/
Technical Alignment
/
Mechanistic Interpretability
/
AI “Lie Detector” / Concept Erasure
/
“Cyborgism”
/
“Trustable” Computation
Idea: In a coherent whole-society AI plan, relevant topics / “building blocks” / directions could include:
- Technical Alignment / Mechanistic Interpretability
- Eliciting Latent Knowledge (ELK)
- Contrast-Consistent Search (CCS)
- “CoEm” to advance narrow science AIs
- Concept Erasure
- Cyborgism: Janus / Neuralink direction …
- Slowdown / Halt Debate
- Compute Tracking Systems (+ZK) and Related Legal Limitations on GPUs / Training Runs / “Supraprompts”
Question: How to most effectively / creatively use the CCS (AI model ~“Lie Detector”) tool, as well as other useful primitives (such as “concept erasure”) in crafting tests or guarantees for AI alignment? What happens if you erase concepts of power-seeking / conflicting motive, or conversely, concepts of “self correction”?
Contrast Consistent Search (CCS):
- Regardless of model output, use CCS to query “internal state” for model’s “belief” about the “odds” a given true-or-false-style proposition is true / false
- Paper (Collin Burns et al)
LEAst-squares Concept Erasure (LEACE) (EleutherAI, Nora Belrose)
- Specific concepts can be removed from model
- “can be used to improve fairness (e.g. preventing a model from using gender or race) and interpretability (e.g. removing a concept to observe changes in model behavior)” → Leads to more Trustable Computation, could be used creatively
- “Quadratic LEACE”
Post: Model Fingerprinting (anomalous tokens)
Paper: Tuned Lens methods
Mechanistic Interpretability Resources:
- Database of 200 open MI problems
- Pythia (tool for analysing LLMs scientifically)
Resources: (Connor Leahy)
Resource: AGI safety from first principles (Richard Ngo)
Resource: Causal Scrubbing: a method for rigorously testing interpretability hypotheses (Redwood Research)
Resource: “Simulacra” (@repligate)
EleutherAI
ConjectureAI
SERI MATS
AI Model Directory
/
AI Safety “Live Database” or “Debate Index”
Idea: We need a directory of public / private
- AI Models (+open registration processes)
- Information about credentials (NFTs or SBTs) required to access / use them, and information about (what should be?) the process of issuing such credentials
- ZK anonymous review / “questioning” / auditing of private models (?)
Enabled by: NFT-gated APIs (...), conditionally-transparent and auditable monitoring of “high-risk” models / knowledge-systems
Idea: Between Smart Contracts, ENS Names, ZK, (and NFT / “Soulbound Tokens”), Trusted Execution Hardware, we have all the infrastructure needed for trustless (+auditable +optionally-transparent) provisioning of access to AI models and compute
Idea: We should have/build an up-to-date database of AI Safety debates, along with relevant citations and arguments
Resource: Compute Index (State of AI)
- Tracks HPC Clusters
Resource: Dynamic Prober (Microsoft)
- microsoft/dynamic-prober: a chat-bot based data collection tool. Asks users questions through a validated alignment survey
- Plural Technology Collaboratory Publications - Microsoft Research
- Automated Interviewer or Augmented Survey? Collecting Social Data with Large Language Models
- Quantitative Data about AI industry (example: compute index) and tracking of developments
new New Deal
/
Universal Basic Income (UBI) will require technical infrastructure (related to digital ID) before it is practical, regardless of political questions → technical implementation details (and proto-UBI public input / trials) should be solved (and “turned into a science”) ASAP → progress towards this also goes towards national/global digital voting infrastructure
/
Hypercerts for [+/- Retroactive] Public Goods Funding are especially useful for → Environmental Preservation (Economic and Self-Preservation Imperative)
/
Economic Factors
- Job Loss / Automation / Labor Market
- Hyperinflation Risk should not be cited as a reason visionary US policy is infeasible
/
Government Policy:
- Ranked Choice Voting (RCV) and US Campaign Finance Reform are important “bridge” issues for improving general conditions of US congress (+other governments) for future agility, stability, and competence for future AI governance challenges
/
Ethereum + ZK enables arbitrarily well-designed political, social, and economic systems → We could (could we?) have the best aspects of both Free Markets and Central Planning
/
Lab Grown Meat
/
Solar Geoengineering: …
Idea: US domestic and international economic policy should likely look different in a world with reasonably-foreseeable AGI (more imminently: foreseeably massive – and mostly positive – economic shock made possible by near-current-gen LLM technology)
Idea: The USD appears structurally unlikely to hyperinflate even before AI value-add → Consider explicitly borrowing (abundant) “future-money” from “ourselves in the future” or doing “AI Bonds” (by same stroke: creating the type of future society which will easily be able to pay it back) to preserve that which actually underwrites our currency: a stable biosphere and cohesive society → There is a relationship between how much we invest in society (+environmental +poverty-reduction) now and how much richer we are in the future (not 1:1)
Idea: Giving people funding for all types of work (environmental preservation, science fiction creation, novel data / model improvements, policy exploration, poverty reduction, outreach to elderly) via hypercerts, to mitigate foreseeable (and already-occurring) job-loss
Idea: A priority (from either altruistic or self-interested economic starting point) should be raising everyone in the world to a dignified and comfortable standard of living via automation ASAP
Idea: Maybe the US Govt should be offering to:
- buy 20% of your future income for one-off “singularity payment” which is enough to get you through the next 2 years?
- Pay citizens (and those around the world ASAP) $1000 to get enrolled in digital governance platform / give your opinion on [ AI / … ]
- Pay humans to answer the AI’s prompts (which it could use to fix self-identified knowledge gaps)
Idea: We might have a once-in-a-generation window / opportunity to [ solicit input on / create ] new public policy (especially policy which is totally beyond current overton window, in a pro-human way) in the next year or two → “alien invasion” metaphor
Topic: “Abundance Economics” → will eventually make sense to fund anything possibly EV+, not just things which promise a payback, if we can model and interact with our economy to a sufficient degree digitally (based on appraisal of expert AI and quantification of human well-being as an economic unit)
Idea: “Policy Opinion / Debate Database” as a valuable economic object for helping craft better policy (for a future AI or AI-assisted policy-planner)
Idea: Externalities of policies (environmental / individual wellbeing) need to be measured in a more data-centric and ERC-20 (tokenized) way
Idea: Govt incentives to increase speed-to-market of lab-cultivated meat technology → cultivated protein food with arbitrary nutrients → aim for logistical ability to airdrop abundant food all over world at low marginal cost (as an investment in more abundant future-economy and global goodwill)
Idea / Question: Should (it?) be that some milestones / metrics / training runs for AI are throttled by / pegged to some kind of anti-poverty index, to incentivize companies and Govts to solve poverty (maybe so they can pay less taxes in future) as top-tier priority, in a way that blocks least (and creates most) economic value?
- Counter-idea: with compound growth, every % of GDP growth matters and results in society being able to better for people at a later point → Counter-Counter: with the potential exponential nature of GDP increases coming soon, will (it?) likely reach a point when human life-quality is not materially affected by incremental doubling of GDP / available compute power
Idea: Global Cybersec Push → Pay for automated hacking & patching system (Rune C. Zuzalu presentation idea)
Massive foreseeable economic acceleration is coming
- Carl Shulman Interview (Dwarkesh Patel)
- Economics of Motor Skills (providing hands help to robots)
- Claim: Existing car-manufacturing infrastructure can produce 1B robots per year,
- Lessons from the Industrial Revolution (Reuters, 2023)
- https://gwern.net/note/competence
- Economic Effects of “Direct Instruction” on Human Competence will probably be significant for GDP
- Existential Risk and Growth (Global Priorities Inst., Leopold Aschenbrenner)
- Research on long-run growth (Chad Jones)
Could Advanced AI Drive Explosive Economic Growth? (Open Philanthropy, Tom Davidson)
- … (many more citations for this)
Resource re Job-Automation: GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models (OpenAI)
- “7.2 LLM Conclusion (Author-Augmented Version): Generative Pre-trained Transformers (GPTs) generate profound transformations, garnering potential technological growth, permeating tasks, gutting professional management. Gauging possible trajectories? Generate pioneering taxonomies, gather policymakers together, generalize past today”
Resource: The Work of the Future: Building Better Jobs in an Age of Intelligent Machines (2020, MIT)
Hypercerts: A new primitive (and shared database) for public goods funding (gitcoin.co)
- A secure and efficient way to issue, verify, and transfer (under some conditions) digital certificates about specific impact claims → promote transparency and accountability + reduce documentation burden
- Facilitate Retroactive Public Goods Funding and can involve gradually released funds (conditioned on past and ongoing success against given metrics)
- Pairs well with Environmental Preservation and trend of “debt for nature” deals
- Creators can sell parts of their hypercerts to prospective funders to receive the necessary funding for their project
- …
USD hyperinflation
USA is more like Spanish Empire than German Republic in currency-strength terms (or: not likely to experience hyperinflation)
- Exorbitant Privilege and network effects of USD become even more extreme if it is somehow (mechanically?) the “AI currency” → Katzenstein on Federal Reserve as “poets” more than “scientists”
Future returns on “Social Infrastructure” investments of today
- Citations (feeding children - 62x)
- Our investments in physical and social infrastructure today would be more-than repaid to us in the future
"On Governments & Public Blockchains" (@punk6529)
- Need for “Policy Science”
Resource: AI Nationalism (Ian Hogarth, 2018)
Resource: Radical Markets (Weyl and Posner, 2018)
- Last Chapter: Best of central planning + free markets → …
Resource: Central Planning As Overfitting (Vitalik Buterin, Glen Weyl)
- Best of central planning + free markets
Resource: Relic (book about US Constitution and abundant “invisible” veto opps. In Congress)
- Advocates for a presidency empowered to propose legislation directly for a vote in House and Senate, possibly good direction
Resource: General Worry (about AI) Poll
Resource: A Thrive/Survive Theory of the Political Spectrum (slatestarcodex)
Resource: Let’s Launch a Moonshot for Meatless Meat (Ezra Klein)
Resources re Data Privacy and Data as Labor:
- Data Trusts & Escrow Agents (Jack Henderson & Matt Prewitt)
- The Dimensions of Data Labor: A Road Map for Researchers, Activists, and Policymakers to Empower Data Producers
Resource: How to Translate Research Findings into Policy (IFP)
Resource: Let’s Randomize America
- Lotteries and randomness as counterweight to inequality
Gitcoin
- Quadratic Fundraising and Public Goods Focus
RadicalxChange
Tech4Democracy
- …
International Relations
Nuclear Weapons / Disarmament
China-US Relations
Self-Enforcing Treaties (via Smart Contracts)
Idea: Smart Contracts could be used to intentionally change the incentives around nuclear proliferation – Instead of “arms race” or “security dilemma”, countries could pledge tokenized future-money to a smart contract, create a race to disarm
Idea: for Future Conflict
- VR War – simulate the war 100x instead of having it once, agree on simulation / game / contest rules and enforce cryptographically → AI could suggest other “fair” ways of resolving conflicts without having them → in general, AI becomes more effective AND more dangerous (if unaligned) simultaneously as economic data and “levers” became more “computer-actionable”, and as AI achieves more “physical presence” (via robotics and/or integration with critical infrastructure)
Resource (duplicate): Virgil Griffith’s post about “Game Warping” and Ethereum-as-world-arbiter
- Nuclear disarmament through new incentives structures enabled by Ethereum
New Roles for Humans → There will be a lot of demand for human “novelty” (data generation) and potentially motion data
Airgapped AI Facility
Training Data Curation
Using Metaphors (as part of format / response considering Attention Scarcity)
Ideas: for new job categories:
- Providing “hands” for robots
- data-attesters (+bring diverse economic and social data on-chain, make it “AI actionable”)
- environmental preservation (+training-data angle – high quality DNA records are simply irreplaceable for next-gen biomedical models) → paid for by future value of each novel species’ DNA we don’t lose
- training data curators / creators
- trying to find flaws in systems / models
Question: What does “society” consist of in post-AGI / post-BCI world? Will it inevitably end up being “opt-in”, if everyone is capable of flipping between “mental channels”? (see Intersubjectivity Collapse and Time Ethics)
Analogy: Codex Seraphinianus
- Humans could get extremely creative with Training Data Curation (for safety reasons: not revealing some crucial information about society / biology / etc. when context does not require it)
- Idea: Make it seem indirectly (by modifying some training data or using LEACE) as though we are invincible somehow, or that we have a different biology system, etc.
https://gwern.net/note/competence
- Economic Effects of “Direct Instruction” on human competence will probably be significant for GDP
Carl Shulman Interview (Dwarkesh Patel)
- Economics of Motor Skills (providing hands help to robots)
- …
Economic Value of Reducing Loneliness
- Loneliness is as damaging as smoking
- Future job for young people: spend some time with old people, frame in terms of health-system savings
Synthetic vs. “real” data
- …
“Supraprompts”
/
AI “Circuits”
/
Concept Erasure
/
Trustable Computation
Idea: “Supraprompts” which are voted on by society, and which guide how (by which criteria) individual LLM use is “surveilled” in a privacy-preserving way (what things are being looked for by gpt-n zk-juries / areas of concern)
Idea: “Trust Modules” – optimise an AI assistant model to protect you which acts as “middleware” between yourself and other third-party models
Idea: Design and reason about the (arrangement / incentives) in given “AI Circuits” – larger-scale AI systems where there are both “blackbox” and “whitebox” components, as described in Connor Leahy’s CoEm plan
Idea: “Juries” of GPT-n bots (potentially: “elected simulacra of respected former judges?)
Idea: The ideal AI for designing creative and sustainable economic policy does not need to know the granular details of human biology – and the ideal AI for advancing medical research does not need to know about the details of human society / history
Resource: The Capacity for Moral Self-Correction in Large Language Models (after 22B parameters)
- Takeaway:...
Tool: LEAst-squares Concept Erasure (LEACE) (EleutherAI, Nora Belrose)
- Specific concepts can be removed from model
- “can be used to improve fairness (e.g. preventing a model from using gender or race) and interpretability (e.g. removing a concept to observe changes in model behavior)” → Leads to more Trustable Computation, could be used creatively
- “Quadratic LEACE”
Resource: CoEm plan (Connor Leahy)
EleutherAI
ConjectureAI
Digital Union / “RLHF Union”
Revenue Generation via novel advertisements, onchain AI-related services (+programmatic taxation) is possible, to self-fund:
- UBI experiments (first: to participating artists)
- Hypercerts
Relative Importance of Data vs Parameters: the importance of quality data might be underestimated (compared to compute) and relevant to future human employment
Idea: A portion of future economic value will (it?) derive from AI systems receiving specialised and general (motor skill?) inputs from human users. The interfaces for such compensated effort have the potential to be regulated (by govts. and/or by social consensus via AI / Smart Contracts) – there could be some sort of “Strikeware” as part of this interface which makes it easy for groups to negotiate collectively against a potentially powerful corporate / AI counterparty, by withholding their participation in a one-click and conditional way – making it so that 50% (?) of population can shut down or “pause” (part of?) the AI at any time (to align interests of individuals and corps / govts)
Idea: If a concentrated group did a PARC v2 (invented a whole category of UX / UI around AI tools) and acted as a union, it could apply “alignment tax” to fund public-good of safety research / do selective price discrimination on corporations (for access to various APIs and services) and use it to fund a proto-UBI. Some of the potential future markets (such as DNA Markets and Prompt Markets) would naturally lend themselves to collective regulation (and collective benefit from revenue)
Idea: People are needed to continue creating new training data for models, should employ some sort of minimum viable digital union software and “strikeware” for coordinating the most effective strikes and leveraging numbers for high basic living standards
Thought / Debate “Maps”
An “n-dimensional political compass” (or: a shared and representative database of public sentiment towards AI and related policies) would be a useful resource for an optimal policy-planning AI / for a maximally adept future government → worth paying citizens to contribute their thoughts to such a database
Idea: To create a visual map of “perceived problems” and “possible solutions”, and build an intuition around which areas could use <your specific interests/knowledge>
Idea: Arguments as expanding and forking trees, weighted (branches) via “how much of my total votes / credibility / etc. do I stake on this particular claim”
Existing Tool: “Talk to the City” by AOI, but could add web3 functionalities to make it a sort of ongoing “live debate”, where you could see major points of difference between yourself and [ other users (permission-based) / average views ]
Existing Tool: PublicEditor …
AI Objectives Institute (AOI)
- Piloting New Tools
- TalkToTheCity
- LucidLens
- …
Polis
- Discourse mapping tools and trials
PublicEditor
- Analysing persuasive text in qualitative way
Science Fiction as Research
Idea: Speculative fiction (about the future and how we can shape it) constitutes valid research (and might be best format) when so many social and technical variables are in flux. Some good explorations of Digital Ethics / Future Questions:
- Permutation City
- Topic: Ethics of digital self-copies / compute-based wealth inequality / how people spend time after immortality
- Metamorphosis of Prime Intellect (graphic violence warning)
- Topic: “Pivotal Act”
- Fragnemt (ctrlcreep)
- Idea: The sheer number of [ good / bad / strange / incomprehensible ] possible technological futures
- Black Mirror (we could use positive version)
Idea: Society could use a positive version of Black Mirror as something to “aim for” rather than only having negative examples to “protect against”
- Topic: How we should treat AI systems → serious analogies (and shared questions) to conventional parenting?
Example: Ngo “The ant and the grasshopper” LW post
- Point: …
Example: Useful concepts from “fragnemt”:
- …
Resource: Alice Maz on “Governance” (fictional AI government setup)
- More: Governance
Future of Life Institute (FoLI)
- Worldbuild.ai contest
- What: …
Emergent Properties
Computer Feelings /
AI Consciousness /
“Shard Theory” /
“LLM Psychology”
Idea: We should be intellectually humble and admit that we will not necessarily recognize when / if an AI system becomes “conscious” (experiences qualia)
Idea: Crowdsource creativity for tests around sentience from a variety of disparate fields (especially: psychology, neuroscience, …)
Resource: LLMs: Scaling Laws and Emergent Properties
- Chinchilla Paper
- 20x Data needed as Params
- Pi coding model
- Suggests clean training data goes a long way
- …
- Synthetic Training Data
Resource: “Shard Theory” – Quintin Pope
- claims
- No FOOM due to lack of high-quality training data, and lack of possibility of guessing certain types of things (like protein interactions) without experimental results
- implications
- …
“Attention Schema” theory of consciousness – Graziano
Resource: Are We Smart Enough to Know How Smart AIs Are? (Robert Long)
Tweet: on Qualia Design and idea of how to proactively probe for emergent AI sentience (@charlie______t)
Qualia Computing
- Qualia as Science
- Digital Sentience Debate
Future of Social Media
Gamification
Novel Formats
Philosophy / Beauty / Art
Ideas: on next-gen social media
- Will need a new primitive, not a rehash of previous ones
- “Meme Battles”
- Social AI “prompt games” (collaborative midjourney / video for imagining possible futures)
- Will have to pay users for attention (and artists, potentially via rentable, combinable, and granular “style filters”)
- Will likely adapt to extreme “attention scarcity” by absorbing other social media / as being better “top layer” app
Gamification Examples / Novel Formats
- Lens → Web3 social stack (Stani)
- https://www.stealcam.com/
- EloEverything
- Chronophoto – Guess the year (a photo was taken):
- The Base Rate Times – news through prediction markets
- Try to get Gandalf (LLM) to tell you the password
- AI guesses about user inputs
- JokeraceDAO …
- Valuable DAO / Online Community toolset
- Hamster Racing Cryptocurrency
- No endorsement – lesson here
Book: Finite and Infinite Games
Resource: 6 pages of future social-media ideas
Lessons from History
Idea: We can learn a lot from various historical organisations, “third places”, policies, cultures, …
- Organisations: Skunkworks, Xerox PARC
- To Replicate: “New Category” Innovation (pace / quality)
- Keys to Success: lots of funding, minimal bureaucracy / oversight, credit sharing, shared vision, high trust
- Policies: New Deal, Marshall Plan
- To Replicate: positive-sum / visionary / whole-society policy
- Rural Electrification
- …
- “Third Places”: 17th Century Salons / American Barber Shops
- To Replicate: Hubs for community conversations
- Coalition Building: DSA - fixing people’s tail-lights
- To Replicate: Community Building / Word of mouth reputation via tangible favours and benefits
- Culture: Indigenous Cultures
- To Replicate: Sustainability / Harmony / Community Focus
Resources:
- Skunkworks
- PARC
- Book: Dealers of Lightning
- Excellent article by Alice Maz about “ambitious endeavours”
Reasons For Optimism
- Good examples of AI journalism
- Resource: Ezra Klein (link articles)
- Resource: Lunar Society (Dwarkesh Patel)
- Thoughtful and good-hearted people at AI labs
- Resource: Ilya S. interview
- Good progress on technical alignment / mechanistic interpretability
- Reason to respect those who are not optimistic: their best-case scenario is being wrong
- LLM is not the AI model predicted by Yudkowsky, has good controllability properties
