Update README.md
Browse files
README.md
CHANGED
|
@@ -9,9 +9,10 @@ This is the sentence-level, supervised, sparse autoencoder (S3AE) proposed in th
|
|
| 9 |
The model was trained on the residual stream in the 10th layer of instruction-tuned [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b-it), using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
|
| 10 |
|
| 11 |
The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
|
| 12 |
-
|
|
|
|
| 13 |
2: 'anhedonia (loss of interest)',
|
| 14 |
-
3: 'pessimism',
|
| 15 |
4: 'guilt',
|
| 16 |
5: 'anxiety',
|
| 17 |
6: 'catastrophic thinking',
|
|
|
|
| 9 |
The model was trained on the residual stream in the 10th layer of instruction-tuned [Gemma 2 27B](https://huggingface.co/google/gemma-2-27b-it), using a proprietary synthetic dataset with psychopathology symptom labels. The model weight precision is bfloat16, and the hidden dimension size is 8 times that of the LLM residual stream.
|
| 10 |
|
| 11 |
The 1st to 17th dimensions of S3AE hidden features, respectively, correspond to activations of the following thoughts:
|
| 12 |
+
|
| 13 |
+
1: 'depressed mood',
|
| 14 |
2: 'anhedonia (loss of interest)',
|
| 15 |
+
3: 'pessimism',
|
| 16 |
4: 'guilt',
|
| 17 |
5: 'anxiety',
|
| 18 |
6: 'catastrophic thinking',
|