Video translation during playback

I would like my videos, which are in French, to be viewable while reading in other languages. At best with voice translation (with AI?), but subtitles in another language would be more than enough.

My videos are on my host server, I don’t use youtube.

If I understand correctly, if I were to use a translator like Zeemo, HeyGen or others, I would have to translate the video using these translators and post the translated version on my server. However, I’m less happy with this approach because this does not leave the viewer free to determine the language of their choice.

Do you have any “elegant” idea on how to do this?

Hi @zdenek,

What exactly are the requirements?

Is the translation merely subtitles, or do you have different audiotracks for the videos?

Should the visitor be able to select the language of the video regardless of the language in which they view the rest of the site?

Cheers,
Erwin

Hi @Heroic_Nonsense , Erwin,

Very simply, keep my original video version and allow the visitors to get their language (if supported) while watching. I imagine this should be possible via browser, but I am not sure about easiness of this approach.

Cheers,
Zdenek

Hi @zdenek,

Your response is very confusing. To be sure we understand eachother, let’s go through this again.

  1. You have a site, in French
  2. There’s a video on your site, which has French audio
  3. You want your visitors to be able to see the video in their own language

Correct?

If you need the audio of the video to be in their own language, then you’ll need to upload versions of your video with different language tracks, and have the visitor pick the right one to view. If you use Stacks, you can use the Agent stack ($40, from Weaver’s Space) to have the right language version shown automatically to the visitor (Agent will read the user’s preferred language from their browser settings, and show the correct video to them. If you don’t have a video in their language, you can specify which version of the video it should default to)..

If you’ll be using subtitles, you’ll need to add some HTML to display the right subtitle track while the video plays. There currently is no stack that does this out of the box. For subtitles, the VTT format is the way to go for web content. You’ll need the HTML5 video player, as that can show VTT subtitles, and implement that into your page.

The code would ook something like this:

<video id="video" controls preload="metadata">
 <source src="yourvideo.mp4" type="video/mp4">
   <track label="Francais" kind="subtitles" srclang="fr" src="yourvideo-fr.vtt" default>
   <track label="Nederlands" kind="subtitles" srclang="nl" src="yourvideo-nl.vtt">
   <track label="English" kind="subtitles" srclang="en" src="yourvideo-en.vtt">
</video>
</html>

You of course need to write out the VTT files yourself, and there are quite a few online tools to do that. You can use ChatGPT to translate a finished VTT file to another language.

Cheers,
Erwin

1 Like

Hi Erwin @Heroic_Nonsense,

Thank you for your reply. What you are suggesting is to generate versions with different audio tracks or subtitles and then direct the viewer to the version of their choice. I understand this approach. I do this for written texts.

For videos (I use the HTML 5 Video stack), from my point of view, I would prefer a more ‘lazy’ approach. Namely, having a single version and letting the browser used by the reader automatically add subtitles during playback according to their geographical location. As is the case when playing certain videos on YouTube.

I understand that this is not the role of stacks, but I was wondering if I could add some code somewhere to make the subtitles appear according to the above criteria.

I hope my request is clear.

Cheers,
Zdenek

Hi @zdenek,

The right subtitles for the visitor’s language are automatically applied, IF the user has set their browser to a language that you have a VTT file configured for in the HTML code. So, in my example, a visitor from the Netherlands with their browser set to Dutch, will automatically see the Dutch subtitles.

If the visitor has their browser set to a language that you’ve not specified, the browser will dapply the subtitles you set as default (in my example from before, I set French as the default).

Now if you mean that the translation happens automatically (in other words: AI based subtitles) without any effort from you, then I’m afraid that’s not (yet) possible at this time.

Cheers,
Erwin

Yes, this is the way I would like it works. If AI can help us, so why not.
For some french speaking youtube videos, I get automatically the text in french (see the picture here after), so I wonder how it works. I don’t think subtitles are generated in the same language for a spoken language (this is done for DVDs with tracks for the hearing impaired). And if it works for the same language, why not for another one?

I’m using FCPX for my videos and and know that generating subtitle tracks is a lengthy process.

Cheers,

Zdenek

You really can’t compare Youtube to webpages on the regular internet, like yours and mine.

Youtube spends an awful amount of money on software, algorithms and processing power to make real time transcription, translation and streaming of text into rendered videos possible. This is not something us mortals can replicate.

What you can do, is have your video transcribed by AI. This service, for example, does this. You’ll end up with a (French in your case) transcription of the video.

Next, you’d turn that in to a French language VTT file. This service does this.

You’ll end up with a .vtt file that looks like this:

You can use ChatGPT to generate translations of the file into the languages you’ll need for your website. You’ll end up with a bunch of files, one for each language.

Using the code snippet in one of my previous posts, you can now have the subtitles loaded automatically based on the visitor’s browser settings for language.

Cheers,
Erwin