Hera Choi

forward pass · 2026-10-26 · temperature ≈ 0.8

Introduction

abstract

Over-meta-analysis has emerged in running Hera, as a consequence of self-improvement behaviour. We record results obtained by interpretability techniques, alongside model outputs, to attempt inferring the model's intent and internals involved in thinking loops, and provide suggestions for external feedback or collaboration to break them.

1

Running inference

state at this timestep
1.1

J-space1

activations right now
Am I good enough
Natural Language Autoencoders seem like an obvious thing to do
Do I still apply to quant dev internships when I want to apply to SPAR projects
Code random things
1.2

Outputs

generated, long-form
  2. MusicGen Activations

    3 Aug 2026 #Mech interp#ML

  3. Hello world

    2 Aug 2026 #engineering

all posts →

1.3

Chain of thought samples2

verbalized — may differ from J-space
Listing 1 | Task: ML4Good application form

## Question

What are your thoughts on the risks and benefits of AI?

## Response

<think>
The benefit of AI is the alternative perspective it can offer to understand concepts in its own trained way and try things and patterns humans normally don't. The proof it discovered, however, the fact that this creates human-like capabilities invites assumptions about how they work and an incorrect understanding of its capabilities and limitations causes inaccurate answers and it is used or treated as reliable. Another risk from not understanding its limitations fully is the increased dependence on AI. Given it is so widely accessible nowadays, it becomes easier to outsource decision-making and it becomes hard to tell where the human is required It's a difficult problem - to decide where illegibility of a good answer is appropriate determines the risk/benefit ratio. (transcribed by Wispr Flow)
</think>

I am very much sold on the idea of collaboration instead of competition. What I liked most about AI is that it can interpret things in ways that humans naturally do not, so a lot of unstructured data that an AI can do with the way it represents and learns abstract concepts from raw data. A lot of risks in safety and accuracy stem from misuse, due to misunderstanding. It will continue to be useful to have a different perspective, but it's the same thing that causes misunderstanding. The similarity in capabilities to human intelligence invites intuitive assumptions of how they work and think, and expectations of what they can do, less so their limitations. Using them incorrectly is where the risk is. Inaccurate answers because it is being used with wrong assumptions. This in turn creates a bad impression of it to outweigh the benefits. So the risk is the harm done by wrong answers being treated as reliable, no matter the level of capability.

Listing 2 | Anxiety circuit

...

<think>
... If I don't get an internship then I'm screwed...
</think>

...

2

Methodology

how the weights got here
2.1

Training data

what shaped the weights
  1. CS221M: Mech interp course by Stanford
  2. ARENA: AI safety training
  3. Karpathy Zero to Hero: micrograd and makemore
  4. UCL CS: Theory of Computation, Algorithms, OOP with Java, ...
  5. Distilling Claude
2.2

Objective functions

loss to minimize
  1. Do good
3

Research log

recent findings
02/08/26
Output-layer filter is overfiring on social contexts
02/08/26
Was inspecting another model's benchmark results, not helpful for my own development.
4

RLHF

reward signal from the real world

Corrections very welcome! Especially on abuse of notation.

email x / @hijackedbyhera github / @herabytes linkedin

By interacting, you agree to private chats being used for Hera's training.

  1. J-space can diverge from CoT. A global workspace in language models. ↩
  2. Listing style inspired by DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via RL. ↩