LLM · Multimodality¶
Review representation, training, inference, evaluation, and safety in vision, audio, and video models.
Start by distinguishing input and output modalities, then examine the responsibilities of encoders, connectors, and language models. For any failure, distinguish lost perceptual information, reasoning errors, and insufficient evidence instead of relying only on an aggregate multimodal score.
Chapters¶
Return to LLM Topics.