A decodability criterion predicts when hidden-state selection beats majority voting in large language models

When a language model generates several answers to the same prompt, the usual way to pick a final response is to count how often each answer appears and choose the most frequent one—majority voting. This works well when the model’s samples are independent, but on hard questions the samples tend to share the same mistake, so the wrong answer can win and adding more samples actually hurts performance. Researchers have tried to look inside the model’s hidden states for a signal of correctness, but the usefulness of that signal changes from model to model and task to task, leaving practitioners without a reliable rule for when to trust it.

What You Need to Know

The paper introduces CASE (Correctness‑Axis SElection), a simple combiner that trains a linear classifier on the hidden state of the token that represents each candidate answer. For every sampled answer, the model extracts the hidden vector associated with the answer token, projects it onto a learned weight vector, and scores the candidate. The answer with the highest score is selected. The training objective is to maximize the correlation between the projected hidden state and the true correctness label, using a small set of verified examples.

Alongside CASE, the authors propose a decodability criterion that estimates how well the hidden‑state space separates correct from incorrect answers without needing labels. It measures the signal‑to‑noise ratio of the correctness axis: the variance of projections for correct answers divided by the variance for incorrect answers. A high ratio indicates that a linear readout can reliably distinguish right from wrong; a low ratio suggests that hidden‑state information is too entangled to be useful.

In experiments across several model families (Llama‑2‑7B, Mistral‑7B, and GPT‑NeoX‑20B) and tasks ranging from arithmetic word problems to commonsense reasoning, CASE outperformed majority voting on the subset of questions where the decodability criterion exceeded a learned threshold. On those hard cases, accuracy improved by 4–9 percentage points, while on easy cases where the criterion was low, CASE fell back to voting‑level performance, avoiding the degradation seen when more samples are added to a flawed majority vote.

Why It Matters

Test‑time computation is increasingly important as models grow larger and deployment budgets tighten. Knowing when to rely on an internal confidence signal lets practitioners spend extra sampling only when it is likely to help, saving compute and latency. The decodability criterion provides a cheap, model‑internal diagnostic that can be computed on‑the‑fly, offering a principled way to switch between voting and hidden‑state selection without external validation data.

Beyond efficiency, the work highlights a fundamental limitation of majority voting: correlated errors can make additional samples harmful. By exposing a measurable property of the model’s representation space that predicts when those correlations break down, the paper gives researchers a concrete target for improving both sampling strategies and the internal structures that encode correctness.

Key Details

• CASE trains a single linear gate per model; no fine‑tuning of the backbone is required.
• The decodability score is computed from a few hundred unlabeled samples per task using the answer‑token hidden states.
• Threshold selection for switching between CASE and voting is done via a small validation set (≤1 % of training data).
• On the GSM8K math benchmark, CASE raised accuracy from 55.2 % (majority vote) to 62.7 % for problems where decodability > 0.38.
• On the CommonsenseQA dataset, the improvement was 4.3 points (from 68.1 % to 72.4 %) under the same condition.
• When decodability fell below the threshold, CASE’s performance matched voting, confirming that the criterion avoids harmful over‑reliance on hidden states.

What’s Next

Future work could extend the decodability measure to other forms of test‑time aggregation, such as weighted voting or ensemble‑based reranking, and explore whether training richer nonlinear gates yields further gains when the linear signal is weak. Investigating how the correctness axis evolves during model scaling or instruction tuning may also reveal ways to design models whose internal representations are more readily decodable, making confidence‑based selection reliable across a broader range of tasks.

📌 Source: Arxiv Ai