Subtitle extraction
Why subtitle extraction needs both the picture and the audio
Hard subtitles, clean dialogue, and missing source text create different evidence paths, so OCR and ASR should not be treated as the same operation.
Key conclusions
- Use OCR when the useful text is visible in the picture.
- Use ASR when the useful evidence is spoken in the audio.
- Keep timing, filtering, and review explicit whichever path starts the workflow.
OCR observes text in frames. It needs region selection, repeated-frame grouping, scene-text filtering, and correction before those observations become subtitle cues. ASR observes speech in the audio track. It needs segmentation, language and speaker handling, noise review, and reading-speed control.
A clean subtitle file may be the best source when it is accurate and authorized. A hard-subtitled master may make OCR the direct route. A clean video with no text may require ASR or a supplied script. A workflow should preserve this origin so the reviewer knows what was observed and what was inferred.
Some projects can use both paths as a cross-check. That does not mean the outputs automatically merge: the team still needs a defined source of truth, timing policy, and correction record.
Start from the evidence in the source
Subtitles may exist as a separate file, as text burned into the picture, or only as spoken dialogue. OCR reads visible text and ASR listens to speech; neither should be selected without inspecting the source condition first.
- Check clean, hard-subtitled, and missing-text cases separately.
- Record whether dialogue, music, effects, and scene text overlap.
- Keep observed text distinct from reconstructed speech.
Use two paths to answer different questions
OCR can preserve wording that is visible even when the audio is unclear. ASR can recover spoken lines that are not visible, but it must handle accent, noise, speakers, and segmentation. When both are available, their disagreement is evidence for review, not an instruction to silently choose one.
| Path | Observes | Needs review for |
|---|---|---|
| OCR | Text in frames | Regions, motion, scene text, timing |
| ASR | Speech in audio | Noise, overlap, speakers, terms |
| Supplied track | Existing timed text | Version, language, alignment |
| Cross-check | Agreement or conflict | Source decision and correction |
Freeze a subtitle-ready source
After extraction, filter, correct, segment, and review the timeline before translation. The source subtitle becomes an upstream asset for every language, so its version and exceptions should remain traceable.
- Inspect picture and audio.
- Select OCR, ASR, supplied track, or a cross-check.
- Extract and time the source cues.
- Correct terms, speakers, false positives, and density.
- Approve the source version before translation.
Choose from the evidence you actually have
A subtitle file contains already-timed text, OCR reads the video picture, and ASR transcribes the audio. These are different evidence sources, so their errors should be diagnosed separately. Do not average OCR and ASR scores as if they measured the same task.
| Input | Method | Output to inspect |
|---|---|---|
| SRT subtitle file | Import timed cues. | Cue text, encoding and timestamps. |
| Words visible in the frame | OCR extraction. | Recognized wording and visual cue boundaries. |
| Spoken dialogue | ASR transcription. | Recognized speech, speakers and timing. |
| Both captions and speech | Select the approved script; compare only when useful. | Resolve discrepancies before translation. |
What if the spoken line and visible subtitle disagree?
An edited subtitle may shorten or paraphrase speech. A voice track may also differ from the written script. Choose the authoritative source for the release instead of merging both automatically.
| Example condition | Decision | Follow-up |
|---|---|---|
| Captions are the approved translation script | Start from OCR or the supplied subtitle file. | Use the audio to inspect timing and context. |
| Audio contains dialogue missing from captions | Start from ASR when the dialogue must be preserved. | Review omitted speech and speaker labels. |
| A product price appears only in the picture | Treat it as on-screen information, not spoken dialogue. | Confirm the local offer separately. |
| Music or overlapping voices obscure speech | Inspect the audio and correct the transcript. | Do not assume OCR can recover words absent from the picture. |
Inspect the source before any downstream production
Before translating or dubbing, play a short representative interval and inspect the extracted text. Correct the first bad artifact; otherwise later stages may reproduce a source mistake with fluent wording. The screenshot below shows asset organization, not a comparison of OCR and ASR accuracy.
Frequently asked questions
Is OCR only for hard subtitles?
No. It can also identify other visible text, but scene text and decorative elements must be filtered before dialogue cues are accepted.
Can ASR replace a missing script without review?
No. It provides a draft source path that still needs speaker, terminology, segmentation, timing, and content review.
Related theme: Localization capabilities
Weydub workflow
Take the next production step
Start with the source condition and choose a focused workflow for subtitle extraction, picture cleanup, or language production.