---
license: mit
library_name: pytorch
tags: [neuroimaging, text-to-brain, scibert]
---
# Text2Brain (mirror)

Unmodified mirror of the released Text2Brain model: Ngo, Nguyen, Chen and Sabuncu, *Text2Brain: Synthesis of Brain
Activation Maps from Free-form Text Query*, MICCAI 2021 ([arXiv:2109.13814](https://arxiv.org/abs/2109.13814)).
Code: https://github.com/ngohgia/text2brain (MIT, Copyright (c) 2021 ngo.h.gia, see LICENSE). We are not the authors. It is mirrored so the NeuroVLM
reviewer analyses can pin it: the checkpoint link of the paper's repository is dead, and the current link is a Google Drive file.

| file | origin | sha256 |
|---|---|---|
| `text2brain_checkpoint.tar.gz` | https://drive.usercontent.google.com/download?id=1mBO1frmlSMgXGN-F1WKQKlCO0gIk_MHq&export=download, downloaded 2026-10-13, byte-identical | `08e8af9af54b5e38e743be56b47c0c987197bc60dc91e3e56aa7feaf406ec6c8` |
| `text2brain.safetensors` | `state_dict` of `text2brain_checkpoint.pth` in the tar (epoch 1220); every tensor unchanged; optimizer and scheduler state dropped | `e1f4dccc5b41d934d0239c5f734a0d05cd61de5ed5ad0162ec2fc5bd2aed43b5` |
| `text2brain-6f29389.tar.gz` | `git archive` of https://github.com/ngohgia/text2brain at commit `6f29389658e23b89e69248f3d3421dd2bd79b269` | `35dbb0e6ea58118727a715bc61ea19c5db70a8a242100d3383af863c452f90b1` |

The model: fine-tuned SciBERT (`allenai/scibert_scivocab_uncased` vocabulary and config) -> Linear(768, 64x4x5x4) -> 3D
transposed-convolution decoder (32 filters), output 40x48x40 placed into the 46x55x46 MNI152 4 mm grid as upstream `predict.py`.

## Running it with current libraries

The upstream code (Python 3.6, torch 1.5.1, transformers 3.3.1) needs two fixes under transformers 4.x:

1. `models/text2brain_model.py`: call the encoder with `return_dict=False`. Otherwise `_, embedding = self.encoder(...)`
   unpacks the keys of the returned `ModelOutput` (`"last_hidden_state"`, `"pooler_output"`) and dropout fails on a string.
2. Drop the checkpoint key `encoder.embeddings.position_ids` (it equals `arange(512)`; transformers no longer saves this
   buffer) before the strict `load_state_dict`. Otherwise the load fails with "Unexpected key(s)" (upstream issue #6).

The upstream code also moves the encoder and the inputs to CUDA whenever a GPU is visible; the NeuroVLM copy keeps them on
the model's device. The patched code is in the NeuroVLM repository under
`scripts/figures/review/core/a1_text_to_brain_baselines/text2brain/`.