syleetolow commited on
Commit
2f667ec
·
verified ·
1 Parent(s): 6f9804b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -2
README.md CHANGED
@@ -9,9 +9,10 @@ This is the sentence-level, supervised, sparse autoencoder (S3AE) proposed in th
9
  The model was trained on the residual stream in the 10th layer of instruction-tuned [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b-it), using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
10
 
11
  The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
12
- 1: 'depressed mood',
 
13
  2: 'anhedonia (loss of interest)',
14
- 3: 'pessimism',
15
  4: 'guilt',
16
  5: 'anxiety',
17
  6: 'catastrophic thinking',
 
9
  The model was trained on the residual stream in the 10th layer of instruction-tuned [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b-it), using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
10
 
11
  The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
12
+
13
+ 1: 'depressed mood',
14
  2: 'anhedonia (loss of interest)',
15
+ 3: 'pessimism',
16
  4: 'guilt',
17
  5: 'anxiety',
18
  6: 'catastrophic thinking',