Chatbots are cool.
We are long past the days when texting a bot was just a novelty, a silly pastime, a way to check your Skype connection. In fact I have this hunch that in the near future many of the mobile apps of today are gonna fully transition into a messenger bot model, especially those that do little more than interface users with a real world service.
As features like live location sharing, payment processing and support for natural language input become more commonplace in the APIs of chat applications, there’s less and less incentive for having a dedicated Uber or Domino’s app taking up your smartphone’s precious resources. As a matter of fact both of these companies have already launched Facebook Messenger bots that do pretty much anything a separate app (or, God forbid, a phone call) could conceivably offer.
And that’s all very well and good, but what gets my inner hacker the most excited about this whole deal is knowing that the tech is getting good fast. Chatbot APIs get better by the minute, and there’s always something new and shiny to tinker with. So that’s what I’m gonna do.
I’ve recently found out about youtube-dl, a neat little command line tool for downloading videos off the web. Looking into it I quickly realized how big of a deal that simple premise was. Besides working with a whole bunch of websites, this little piece of software also came in the form of a Python package that could be easily hooked up to whatever else I felt like. Like, say, a bot.
That’s where I got the idea for Soundbot: a small chatbot assistant that would sit in my contact list, to whom I could forward cool public domain and creative commons media I happened to find online, to be downloaded to my personal music collection.
It would work as follows:
From my phone, while listening to a song or watching a music video that I like, I hit “share”, and pick Soundbot from my contact list.
The bot’s back-end receives the URL of the aforementioned piece of media, runs it through youtube-dl and performs any file processing or format conversion necessary to spit out a nice little mp3 track.
The resulting file is uploaded to a convenient cloud hosting service, such as Google Play Music or Dropbox.
Back on my phone, a cloud-enabled music player detects the change, then syncs and downloads the freshly ripped song.
Once this plan was laid out, I started messing around with youtube-dl immediately, and setting up the whole mp3 ripping part of the project proved surprisingly easy! Here’s how I do it:
from youtube_dl import YoutubeDL, postprocessor
Pretty straightforward, huh? All this function does is create a dictionary of “options”, which describe how the
YoutubeDL object should handle downloaded files. There are only three options in use here:
outtmpl: the template string for the output filename. The value
"out/%(title)s.%(ext)s"means the file will be stored in the
outfolder, its name will be the title of the original source material, and its extension will be whatever comes the other end, as defined by the other options, or youtube-dl’s default behavior.
noplaylist: whether to download every file or just the first one when the source URL points to a playlist.
Truehere means “only download the first thing you find”.
postprocessors: now, this is the juicy part. Youtube-dl comes with a bunch of post-processing options for after the audio/video file has been downloaded. Here all I’m using is the
FFmpegExtractAudiopost-processor so that whatever comes in from the source URL is converted to an mp3 file, as per the
These options are then passed to the
YoutubeDL object, which operates in a handy context manager syntax.
And just like that, in 16 lines of code, we’ve got ourselves a function that, when given an URL to any of the 1089 supported websites, will download the media file, extract its audio track, and save it in a folder. Problem solved!
“But wait!”, you might be saying. “That’s not what we wanted at all!”
And you would be right, my imaginary projection of a reader. The ideal scenario includes not only downloading the desired track, but also uploading it to a file hosting service from where we can retrieve it at a later date. Hmm, if only there was a straightforward way to add an extra step to the file download pipeline. Some way to further process it, a bit after the previous steps were done. If only…
Youtube-dl post-processors are not only super cool and useful, they are also pretty extensible! So, by making use of the
PostProcessor base class, and the Dropbox for Python SDK, we could write our very own post-processor that takes the output mp3 file and uploads it to an easily accessible cloud folder. This is how that looks like:
There’s quite a bit going on here, so let’s break it down. In this file I’m defining the
DboxPP class, that extends the base
PostProcessor class from youtube-dl. We can later use this much in the same way we did
FFmpegExtractAudio, by plopping it into the
postprocessors list in the downloader options.
First off there’s
__init__(), the constructor. This method receives two parameters,
downloader, which is the downloader object created in the
with block in the previous file; and
token, a custom parameter to be defined within the options dictionary. This token is our Dropbox API authentication token, needed to programmatically upload files. To get one of those, read up on how to create a Dropbox app.
Then there’s the
run() method is what’s effectively called when a post-processor starts. It receives the
information parameter: a dictionary of metadata from the downloaded file. It contains a bunch of useful, well, information, but for now the
"filepath" is all that matters to us. This contains the path to the resulting file after all the previous post-processors have done their thing.
upload(). This method takes a file path, reads its contents and passes them to the
Dropbox.file_upload() method, overwriting any existing file, if necessary.
Now that we have our post-processor, it’s time to go back and make some changes to our
rip_audio() function. Let’s see how that looks like:
from youtube_dl import YoutubeDL, postprocessor
First we have to add a couple lines in the beginning to read the Dropbox token from somewhere, no big deal. Now, this next part might look weird at first, and that’s because it really is. You see, the YoutubeDL object expects the list of post-processors to be formatted so that each entry contains a dictionary with a
"key" item, containing the name of the post-processor’s class as a string, minus the “PP” suffix, plus items for any extra parameters the post-processor’s constructor receives, after the
downloader param. Confusing enough?
On top of that, it then uses that string to look the post-processor up inside the
youtube_dl.postprocessor namespace. So we have to inject our class into that by doing
postprocessor.DboxPP = DboxPP.
And just like that, we’re done! Well, sorta. This isn’t a bot at all yet, it’s just a python script that I have to call by hand to download some music for me. But since this post is already long enough, I’ll go into the details of how I’m thinking about building the bot’s back-end and messenger integration later on.
In the meantime, feel free to browse Soundbot’s git repo to watch me struggle through this project in real time. See ya!