Quick Takes

the following fictional dialogue is a complete unapologetic strawman but it's funny enough i had to bring it into being: ---------------------------------------- “So I asked myself: where can I make the most impact? And clearly malaria is the most important area.” “And so you decided to donate all of your money to buy malaria nets?” “Well, so it turns out that saving lives from malaria is actually kind of expensive and indirect. You see, it costs thousands of dollars to save a life. Statistically. Who knows if you’re actually changing anyone's life that way?” “And so you found a more efficient way to save lives.” “Actually, it turns out that it’s cheaper to give people malaria. It's a lot more impactful and the technical problems are more interesting.” "I see. Isn't more malaria bad though?" "I don't know, but I find it much easier to work on because the feedback loops are much tighter. Maybe one day, if malaria gets big enough, I’ll go work on saving people from malaria. But we're still a long way away from everyone having malaria."

When Elon says stuff like "We're in a simulation, and the most entertaining outcome is the most likely", I think we should probably take him at his word. Think about what it must be like to be Elon. You are literally 4x richer than any other person on the planet. How likely actually is it that you are the real Elon Musk, vs. just some dude in the future who for whatever reason got into a sim, so he could experience what it was to be a tech emperor in the past? It would be hard not to put significant probability on the possibility that this is just some Total Recall situation and that none of us are actually real.

I've felt for a little while now that steganography, in the sense of text that looks normal and monitorable to humans but actually allows LLMs to do general-purpose hidden reasoning, is a less likely failure mode of CoT monitoring than switching to recurrent latent reasoning (neuralese) or obvious and visible linguistic drift. I think I have a new and fairly crisp argument for this: Using general-purpose steganographic encoding and decoding itself requires a lot of serial reasoning. For example, one way you can do steganography is by partitioning the set of all tokens into disjoint groups, with synonyms placed in different sets, and every time you have multiple words to choose from with the same meaning, you encode some additional hidden meaning based on which set the word you output belongs to. To reliably output the right token to encode meaning by such a scheme, you probably need to think pretty carefully about how to follow some rules about which tokens are in which set and what you are trying to encode. Similarly, extracting the hidden message from the text requires carefully pulling out the hidden bits and then piecing together what they mean. So either the serial reasoning required for encoding and decoding should be easily monitorable, or there is some non-steganographic reason why that serial reasoning is unmonitorable. Either way, steganography isn't the first reason why CoT monitoring fails. I would guess this is true of all general-purpose steganographic encoding and decoding schemes because it seems hard to hide arbitrary messages in normal-looking text without thinking hard about what to write, but I'm not super confident about that.

Marcus Olang's I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me is interesting because my pre-university education mirrored his due to shared British colonial history, so it brought back a lot of memories: [...] So, story time: I went to the top government boarding school in my country by average GPA in our equivalent of the O-Levels / GCSE. My school, which only had 4th and 5th form students (ages 16 and 17 respectively), was a benchmaxxing factory; the school average GPA was effectively the overriding cost function; all 60ish govt boarding schools were ranked nationally by this, and for the trial exams (but not the real final nationwide exam) we knew not only every school's rank nationally but every student's rank nationally across all boarding schools, all 8,000+ of us. Roughly half the teachers in my school were Excellent Teachers, a literal translation of an official designation given to something like the top 1% of teachers nationwide in one of the standardised subjects as evaluated by (if I recall correctly) student average GPA on our GCSE equivalent in their subject of specialisation. Every student in my school was a straight-A student in the Lower Certificate of Education (LCE), another nationwide standardised exam taken by all 3rd form students. The results spoke for themselves: something like 1-2% of the half-million-odd students nationwide sitting the GCSE equivalent got straight As back in my day while over 60% of us did; ~0.1% got straight A+ while 3-5% of us did, so ~30x base rates. The top students got full-ride scholarships sponsored by either the government, the central bank, or government-linked O&G companies to study abroad in the UK, Australia, North America etc; this was life-changing for us. A lot of my peers came from rural areas and didn't really have good English. They did have great short-term memory, tremendous work ethic and pain tolerance, and a really intense desire to get those life-changing scholarships, because pov

what's the strongest argument for why i shouldn't auto-ignore any acausal arguments that involve hypothetical entities extremely far away (or which only exist in other Everett branches or whatever) such that we will never interact with them causally at all? a razor i have is things which are entirely epiphenomenal should be ignored because they are unfalsifiable. in particular, this seems consistent even if you accept one-boxing and paying in the counterfactual mugging. the key question is what kinds of evidence you accept as evidence of the existence of an acausal connection. in these hypotheticals, we simply declare by assumption that Omega is truthful and capable of predicting you. in reality, we would either arrive at such a belief by empirical observation (Omega has a strong track record), or pure theoretical deduction. all of the epiphenomenal acausal theories depend on pure deduction. that empirical observation has to be conveyed to us causally. it seems reasonable to draw a line and say we simply don't trust pure deduction to be able to convince us of an acausal link. also, with both the counterfactual mugging and Newcomb, even though in each instance you can't prove the other branch could have happened, in the long run those who do the right thing (pay, one box) will win. whereas with purely epiphenomenal acausal theories you will literally never find out any difference whatsoever, because the branching point happened long ago.

Here are some of my top candidates for big pushes to do right now on technical AI safety (low effort notes): * Much better model organisms / misalignment analogies: * Doing a wider set of pessimized training runs * Good candidate for lots of AI labor automation? Like maybe good to try to set up pipelines for building these envs. * Demonstrating risks from fitness-seekers/reward-seekers empirically * Even on current models with better tests, see here * Demonstrating various types of memetic spread of misalignment? * Actually do control * Build pipelines for red-teaming monitors and the agent itself. For the agent red-teaming, I'd put particular focus on checking whether it continues malign trajectories. * Scaffold integrated control features and other non-monitoring runtime control measures * Human response and auditing * Improving async and sync monitoring * Agent security features * Surveilling for rogue internal deployments (as in, building after-the-fact detection methods for rogue deployments) * Preparing for handoff and elicitation * Get AIs generically better at conceptual work * Have a plan for the evals we ultimately need to see if handoff/deference would go well and start iterating on earlier versions * These presumably will involve a bunch of manual scoring, so we'll need to build a process for it. * Analyze AI biases and epistemics and improve across many domains * Build the anti-slop/anti-mundane-misalignment coalition via doing ratings of AIs and applying some pressure to improve on these ratings. This could focus on a variety of related issues. * The hope is basically that there might be widespread interest in removing/redacting mundane misalignment and other non-misalignment behavioral problems that reduce productivity and large parts of this seem differentially good. So, if we could make this a salient metric, AI companies might improve this. A lot of the difficulty would be in measuring

Anthropic's recent paper on Natural Language Autoencoders trains an activation verbalizer (AV) and activation reconstructor (AR) jointly via RL to reconstruct a target model's activations through a natural-language bottleneck. The hope is that the bottleneck text is human-interpretable. A concern: since AV, AR, and target are all Claude, they share representational priors. A misaligned model could just learn "unnecessary adjective in sentence two = secret desire to kill all humans". This gives great construction loss, and hides misalignment. The paper does test for this post-hoc: they apply meaning-preserving transforms to AV outputs and check whether their reconstruction score collapses. It doesn't, but they themselves acknowledge that this may not hold under optimization pressure. My Proposal: rather than using paraphrase only as a probe, put it in the training loop: pass AV outputs through an unrelated paraphraser (non-Claude, or an ensemble of unrelated models) before the AR. This should be cheap relative to the existing two-model RL, and using multiple unrelated paraphrasers would force the AV/AR pair to be robust to writing styles outside Claude's prior. I think it's very unlikely that a secret misalignment could reliably be smuggled through multiple different paraphrasers, which means that optimization pressure would incentivize the model to avoid steganography. If an already-misaligned model was trained this way, it would have to choose between "keep my misalignment hidden" and "get a higher reward" and eventually the higher reward might win out.