I'm going to make a browser extension that can right-click-save *any* web video, even the ones streamed in segments. There's a reason browsers don't have a button for that already and the reason is that it's hard, so I'm probably going to go insane. - ThreadSky

retr0.id • 123 days ago

I'm going to make a browser extension that can right-click-save *any* web video, even the ones streamed in segments.

There's a reason browsers don't have a button for that already and the reason is that it's hard, so I'm probably going to go insane.

Comments

gebi.bsky.social•122 days ago

That sounds awesome! As for yt, tubearchivist and their extensions comes close, for yt at least. Injects a download button on yt website and exports session to server for download and archiving.

ejsponge61.com•123 days ago

Maybe use https://cobalt.tools as a base for that somehow

waxmonkey.bsky.social•123 days ago

bobbydigitales.anderberg.co.uk•113 days ago

yt-dlp already does this at the command-line. A browser extension would be super awesome!

retr0.id•113 days ago

yt-dlp only supports a subset of websites

bobbydigitales.anderberg.co.uk•113 days ago

1827 sites is quite a lot! https://github.com/yt-dlp/yt-dlp/blob/master/supportedsites.md

retr0.id•113 days ago

not as a percentage lol

retr0.id•123 days ago

Google will also get mad at me because it will work for youtube and they don't like it when people can download youtube videos

meson.ninja•123 days ago

think they'd get as mad about a playlist generator for external media players

retr0.id•123 days ago

I think they would if they could. iiuc they tried to go after ytdl on the basis of it (supposedly) circumventing a TPM, but they can't do that if there's no TPM being circumvented.

retr0.id•123 days ago

the hardest part of all is that it's going to involve writing javascript

retr0.id•123 days ago

I think, in theory, this is the only interface I need to hook. I love interfaces.

The surface area looks small but it's deceptive, there's a lot of complexity here.

retr0.id•123 days ago

hmmm probably the MediaSource interface, too.

remuxing separate audio/video channels is going to be a pain. especially if I want to think about handling mid-stream resolution/codec switches (can container formats even represent such things?). I'll leave this out of the MVP though.

grumpy-dev.bsky.social•123 days ago

Matroska (mkv) containers might be able to, but I have never tried it.

retr0.id•123 days ago

Oh no there are *four* different MSE bytestream formats I'll need to support

https://www.w3.org/TR/mse-byte-stream-format-registry/#registry

retr0.id•123 days ago

And this isn't a case of "there are four standards but everyone just uses one of them", I'm pretty sure all four are in active use

eyebrowsofpower.bsky.social•123 days ago

Hmm, can you save it as HLS? Most annoying format on disk, but can be re packaged later?

retr0.id•123 days ago

not trivially because a) not all streams are HLS b) the browser does not natively consume HLS

wills.co.tt•123 days ago

https://github.com/ffmpegwasm/ffmpeg.wasm exists - wonder if the easiest route is an in-browser version of yt-dlp?

retr0.id•123 days ago

I'm uninterested in re-encoding, and while ffmpeg might help with remuxing it would be a huge overhead for something that is (in principle) doable efficiently in vanilla js

romabysen.bsky.social•123 days ago

I am so sorry

floriandev.bsky.social•122 days ago

You could try and compile ffmpeg to wasm. Then link it into your own binary and then maybe write less javascript. Though interacting with websites is probably much simpler with javascript

wang.social•123 days ago

You’re also going to have to work around some pretty severe DRM logic in any of the commercial video playback sites, and possibly even into the browser itself. You can probably do anything with a browser extension but do expect it to get pretty gnarly. JavaScript is the least of your worries.

retr0.id•123 days ago

dw I already wrote a paper on that ;) https://www.theregister.com/2024/09/12/cenc_encryption_stream_attack/

retr0.id•123 days ago

But, for the purposes of this extension, I consider DRM out-of-scope. It'll just download the streams in encrypted form.

wang.social•123 days ago

Hot damn. Awesome

theteapartyfan.bsky.social•123 days ago

Icky

rfksbrainworm.bsky.social•123 days ago

I know you know about this, but PSA for those who might not

blackstonecode.bsky.social•121 days ago

Between yt-dlp and Video DownloadHelper I'm able to get pretty much anything I want including video that streams in segments. I imagine David is more interested in hacking on his own but I thought I'd throw it out there.

orpach.bsky.social•113 days ago

youtube-dl/forks rely on extractors to get the URLs it needs to download. Most are site specific, some generic.

This extension, as described, would be able to nab anything that the browser can play. Different approach, really.

If it ends up existing, I look forward to a ytdl extractor using it. :)

jamie-dixon.co.uk•123 days ago

I use ffmpeg to download those videos streamed in segments. Could probably push this to the server and do the same from a plugin. Happy to share the little script I use for doing it with ffmpeg.

retr0.id•123 days ago

I assume this involves parsing an m3u8 or similar? My intended approach is fully generic.

jamie-dixon.co.uk•123 days ago

Ah yes. I usually pass the url of the m3u8 and let ffmpeg do the rest.

moll.dev•123 days ago

I've been trying to startup up some kind of decentralized internet archive (maybe using IPFS). I have a local setup where every link in my Obsidian notes gets archived via archive box onto my nas (and the link points to my local copy)

benoist.dev•113 days ago

I’ve been wanting to set up something very similar. Right now I use syncthing for my written notes, but would love a way to archive articles and such that I find interesting. Bookmarks don’t last.

cathectic.bsky.social•122 days ago

Here's more power to your elbow! 💪

zchris13.bsky.social•123 days ago

doesn't yt-dlp do this already

retr0.id•123 days ago

only for the finite set of sites it has up-to-date extractors for

zchris13.bsky.social•123 days ago

ah, yeah, needing to rename X links to twitter when I feed them to yt-dlp is pain

hiccup.bsky.social•123 days ago

That's done per-site too:
https://bsky.app/profile/retr0.id/post/3l7iu456qzx2k

I think the idea with this is to make use of the fact that a lot of video playback in sites now is pretty standardised.

hiccup.bsky.social•123 days ago

Although there are advantages to having per-site code, e.g. you can access more data by using APIs directly, rather than just what is exposed via the web interface.

watchiyourieyes.bsky.social•123 days ago

I fully support you in your endeavor.

angiestone.graysky.social•122 days ago

+ 1!

vai.noetic.work•116 days ago

arms race -> stream embedded advertising -> everyone cries?

retr0.id•116 days ago

We already have stream-embedded advertising, which is defeated by tools like sponsorblock. Combine this with avcut (https://github.com/anyc/avcut) and you have ad-free downloaded videos.

vai.noetic.work•116 days ago

I can just see it changing, they'll randomize the ad locations, etc etc. Sponsorblock still depends on crowdsourced data atm. I'm all for you doing this :) I'm also always interested in the network effects of capability/tech shifts, and in particular, on their adoption.

cheval1.bsky.social•123 days ago

Godsent

taurean.posts.cv•122 days ago

That sounds extremely useful. I hate spending a >$200 for a web course that I can lose access to

512.bsky.social•114 days ago

a lot of those courses have DRM though. easy to break DRM (assuming that it's widevine and not something else), but still legally questionable

smilerryan.com•113 days ago

The closest I know other than a few select specific software is just to screen record them. i've figure out how to get the pssh key and licence request(?) but I don't know how to turn that into a decryption key to decrypt the downloaded mp4.

retr0.id•113 days ago

pssh is not a key, but metadata ("Protection System Specific Header"), usually containing key ids, and it's used to form the license request.

retr0.id•113 days ago

the CDM is responsible for generating the license request, which is sent off to the licensing server by the browser, which returns a license response (slight simplification)

the CDM uses the response to derive the required key material, using whitebox crypto and/or hardware security features

retr0.id•113 days ago

the derived key material - usually symmetric "content keys" - is held within the CDM, and is used to decrypt the media using, again, either whitebox crypto or hardware security features

gyrfalcon05.bsky.social•123 days ago

very interested to see this in action

sidewalkslam.bsky.social•123 days ago

I'm going to tell you this is basically impossible and will drive you mad. Take it either as a warning to save yourself stress or as motivation to tell me to eat shit when you get it done.

michelbestaat.thereforeiam.eu•123 days ago

Cool. I can save the video's from Bluesky from my Android app.

marko.tech•123 days ago

https://github.com/yt-dlp/yt-dlp that'd be useful - it's quite doable; but make sure processing handled clientside. probably will get takedown requested out the ass i'd imagine

plourples.com•122 days ago

don't tv channels have official ways to watch them online now? bbc or channel 4 in the UK at least. this just seems like easy piracy and i worry if you'll be ok

smilerryan.com•113 days ago

as far as I know for channel for all the content on their site is free but may or may not at some point be taken down.

jaz.bsky.social•123 days ago

I'm sure you're familiar but cobalt tools may have some good prior art to reference - https://github.com/imputnet/cobalt

retr0.id•123 days ago

Does it do MSE hooking?

jaz.bsky.social•123 days ago

I'm not sure, haven't looked tbh

diaeter.bsky.social•123 days ago

I assume you're right and wish good luck,seconding the need for such an extension.

liarspeaksthetruth.bsky.social•122 days ago

Use OBS and screen record any video you want whenever you want.

retr0.id•122 days ago

this implies a re-encode which is gross, and limits you to real-time recording only

gadgetoid.com•123 days ago

Delving into developer tools trying to find the damned video source is perhaps my least favourite game- why is it all so cursed 😭

baameows.gay•123 days ago

i love you

neinmc.bsky.social•122 days ago

worst case scenario, you could just compile to yt-dlp and ffmpeg to web assembly, so the user only has to download all that stuff and then wait for 15 hours while the fans spin

chadloder.bsky.social•123 days ago

Take a look at yt-dlp. The logic there is what you really want to duplicate.

retr0.id•123 days ago

Not really, yt-dlp is not generic

chadloder.bsky.social•123 days ago

yt-dlp has extractors for a large number of different platforms and formats. It's more useful than a one-save-per-click extension IMHO

retr0.id•123 days ago

I specifically want to cover the use-cases that the likes of yt-dlp do not cover

retr0.id•123 days ago

Also, yt-dlp frequently breaks due to its inherent adversarial relationship with platforms like youtube. My intended approach can't be blocked for as long as browsers are still able to play the video.

chadloder.bsky.social•123 days ago

Makes sense!

sector.bsky.social•122 days ago

please do go through with this 🙏

nahkd.bsky.social•123 days ago

i have a tip for ya: since most videos on the web are HTMLVideoElement, you can make a new MediaStream from it then pipe it to MediaRecorder. not perfect but i think it will work. or try obtaining the stream directly from srcObject. or maybe try the canvas2d trickery, idk.

retr0.id•123 days ago

TIL. This implies a re-encode though, right? I want to avoid that at all costs

smilerryan.com•123 days ago

I would assume you can pull from the beginning to the end directly without re-encoding by just keeping the video in ram or disk whilst saving it. For now even if it's not directly playable having a downloadable stream that can be passed into a tool to add the header back in it would probably be fine

retr0.id•123 days ago

(and presumably it won't let you do that for a DRM'd video?)

nahkd.bsky.social•123 days ago

i've not tested video with DRM protection, but in my theory, all the DRM stuffs are probably handled by your display/gpu anyways so it's quite hard to capture the content with just software alone.

smilerryan.com•123 days ago

Disabling hardware acceleration in chrome/Firefox let's normal programs screenshot so for that type of content you can just tell users to force software rendering.
Wether this is helpful or not I don't know but opera gx has an AI enhancer for YouTube, and on movies using it lets capturing it work.

nahkd.bsky.social•123 days ago

yeah unfortunately it is reencoding the video. i also don't think there is a way to extract encoded video frames from HTMLVideoElement using web api either; maybe you can grab the chunk data with extensions api, like getting them from url of the stream that you've obtained from extension for example

videocrime.48minutesofdogsbarking.com•123 days ago

Before you do, check out https://cobalt.tools

retr0.id•123 days ago

I'm aware, but like 99% of existing video download tools, it requires support to be added on a per-site basis.

videocrime.48minutesofdogsbarking.com•123 days ago

Point taken

dsamblas.bsky.social•116 days ago

📌

justinmaurer.dev•123 days ago

👀

jesniha.bsky.social•123 days ago

I'll be waiting, no pressure

khide.bsky.social•122 days ago

This would be dope 🙏

jafupy.com•121 days ago

Followed for news

nocab.lol•116 days ago

i assume this excludes widevine videos lol

retr0.id•116 days ago

I'll include a decryption option for power-users who know how to resolve keyids

retr0.id•116 days ago

and/or an option to download the encrypted version with sufficient metadata attached to decrypt it later (e.g. via DeCENC)

nocab.lol•116 days ago

oooh fun!

xeiaso.net•123 days ago

You're already insane :)

lividcola.bsky.social•123 days ago

and that's why I use yt-dlp

cmcalgary.bsky.social•122 days ago

🙏🙏🙏

elderofancients.bsky.social•123 days ago

Uh.... That's gonna be hard unless you install a driver and can just cap the screen or bid buffer directly

vaalgamon.bsky.social•122 days ago

I think "any" is kind of a lofty goal you'll have to deal with DRM'd video like Netflix. I know you said cobalt isn't what you want because sites have to be added individually but it's probably the best way to do things.

isthere.an.echo.inhere.orisitjust.me•123 days ago

i will install it immediately

ywle.bsky.social•122 days ago

easy way to get yourself blacklisted from every extension store
also drm is unavoidable

domi.zip•123 days ago

How do you plan to bypass all the right click event hooks that prevent browser menu without impacting website functionalities? I'm just curious if you can think of any way that isn't just activating the extension for a click.

retr0.id•123 days ago

hold shift

domi.zip•123 days ago

til

that's useful

retr0.id•123 days ago

idk if it works on chrome too but it definitely works in Firefox

clairethephantom.bsky.social•123 days ago

It works on Edge so it most likely works on Chrome

sergebenard.com•122 days ago

I've been wondering about this days ago when I saw a cool video on BSky and saving it was a roundabout affair

I'm super interested in what you find out!!

edgygamergal.com•123 days ago

I am saluting you in your efforts and here to watch the delve into madness

andres.rive.ro•122 days ago

I know you seem to want this to be lower level but I wonder if this kind of extension would be best served like ad block, where users submit per website javascript and the elements to process for each website.

nichromekneesocks.bsky.social•123 days ago

Its not actually hard they just don't want to enable downloading and get punished for it by web publishers

retr0.id•123 days ago

false

bnuuy.bsky.social•123 days ago

why is it false

clockworklady.bsky.social•123 days ago

Because it's not true?

bnuuy.bsky.social•123 days ago

y-yes, that is why i'm asking. i want to know why it's false, why it isn't easy to download videos

retr0.id•123 days ago

If you read the rest of my thread, you'll get some idea of it.

Streamed video isn't just "here's the video URL, play it", it's an incredibly complex state machine.