Multimodal training data
Multimodal training data is a set of paired text, image, audio and video examples used to train models that work across more than one medium.
Last reviewed
What multimodal training data contains
- Image and caption pairs
- Video with transcripts
- Audio with text
- Cross-modal reasoning tasks
Who buys it and how it is priced
| Attribute | Value |
|---|---|
| Belongs to | training data |
| Main buyers | Labs building vision, voice and omni models |
| Pricing | Priced per item or per hour of media, with higher rates for dense human annotation. |
Quality criteria and risks
Quality criteria
- Alignment between modalities
- Annotation density
- Rights clearance
- Diversity
Risks
- Mismatched captions
- Copyright in images
- Biometric data
Providers that list multimodal training data
Auction and pricing mechanisms that fit
- Posted-price sale: Fits catalogue datasets that can be sold many times, such as cloud data exchange listings.
- Reverse auction: Fits a buyer that knows what it needs and wants several vendors to compete, such as sourcing expert data from labs.
Key terms
- annotation
- Annotation is the process of adding labels, ratings or structure to raw data so a model can learn from it.
- data provenance
- Data provenance is the documented history of where data came from, who created it and what rights attach to it.
- data licensing
- Data licensing is the grant of rights to use a dataset for stated purposes, in return for payment or other terms.
Frequently asked questions
- What makes data multimodal?
- It links two or more media types, such as an image and its description, so a model learns how they relate.