| 
							 | 
						--- | 
					
					
						
						| 
							 | 
						description: Learn about Ultralytics transformer encoder, layer, MLP block, LayerNorm2d and the deformable transformer decoder layer. Expand your understanding of these crucial AI modules. | 
					
					
						
						| 
							 | 
						keywords: Ultralytics, Ultralytics documentation, TransformerEncoderLayer, TransformerLayer, MLPBlock, LayerNorm2d, DeformableTransformerDecoderLayer | 
					
					
						
						| 
							 | 
						--- | 
					
					
						
						| 
							 | 
						 | 
					
					
						
						| 
							 | 
						# Reference for `ultralytics/nn/modules/transformer.py` | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						!!! Note | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						    This file is available at [https://github.com/ultralytics/ultralytics/blob/main/ultralytics/nn/modules/transformer.py](https://github.com/ultralytics/ultralytics/blob/main/ultralytics/nn/modules/transformer.py). If you spot a problem please help fix it by [contributing](https://docs.ultralytics.com/help/contributing/) a [Pull Request](https://github.com/ultralytics/ultralytics/edit/main/ultralytics/nn/modules/transformer.py) π οΈ. Thank you π! | 
					
					
						
						| 
							 | 
						 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.TransformerEncoderLayer | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.AIFI | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.TransformerLayer | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.TransformerBlock | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.MLPBlock | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.MLP | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.LayerNorm2d | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.MSDeformAttn | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.DeformableTransformerDecoderLayer | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						## ::: ultralytics.nn.modules.transformer.DeformableTransformerDecoder | 
					
					
						
						| 
							 | 
						
 | 
					
					
						
						| 
							 | 
						<br><br> | 
					
					
						
						| 
							 | 
						
 |