Speed up your Podcast Editing with Generative AI (Developer Advocate Version)

For the first two seasons we used Riverside.fm as they offer a generous two hour recording limit for separated tracks.

They also offer some unique features like Magic-Clips creation.

Overall I liked the experience as the setup was easy. If you also want to edit inside a browser and get a transcript for free, Riverside.fm is your place to go.

The only downside I found and that might be specific to me was that the exported quality lacked. At least for my voice I had overly loud white noise and also the sound level was way to high in every recording. By cleaning it up my voice suffered which is a real bummer as the sound quality is excellent when I record locally 😕.

Streamyard — as the name implies — is designed for streaming. But it also has a recording feature which records locally.

Setup is fairly easy and straightforward making it an alternative to Riverside.fm if you do not need all the fancy features.

Also the audio quality seems to be a tick better in my opinion. And actually that is all I care about!

Disclaimer: I did not try Squadcast yet as I exhausted my free minutes with a transcription 🙃

Squadcast looks like Riverside.fm and Streamyard when it comes to features. The real gamechanger comes with the integration of Descript (See later in the article).

Having a recording solution where you do not have to import the recording into another tool looks like a no brainer to me.

Auphonic looks extremely strong when you look at the features. Except from removing filler words it should be your one-shot solution to produce great audio.

Also the demos are impressive and I could not wait to try it out.

But unfortunately I could not make it work in my case. I experienced the following issues:

What worked well:

Adobe Podcaster offers a free audio enhancer where you can remove noise and echo from your audio, but not filler sounds and stuttering.

For me it did fail on:

But it did a great job enhancing the audio. Both voices felt rich after enhancing.

AI-coustics did a great job at removing noise. It was gone and the voices still sounded nice.

The only problem I noticed was that sometimes the volume levels dipped a little bit.

If you have heavy background noise I would probably use AI-coustics as in my opinion it did the best job with noise removal.

Audo.AI did a similar good job with the audio quality like AI-coustics. But a little bit of background noise was noticable.

If I would create video or audio content every week I would definitely purchase Descript!

The editing is a completely different experience: You edit on the transcript! And then the audio adapts to that 🤯.

Removing specific filler words is a Search+Replace away. Getting rid of silence is also well done: You specify the gap width that should be shortened and to what it should be reduced.

Also the sound quality is great. No background noise hearable in my case.

For now I stuck with Cleanvoice because they have a credit based payment model. I purchased the credits and can use it any time I want.

The best thing about Cleanvoice is that it removes filler words, stuttering, background noise and silence automatically. Although you still have to edit a little bit, because automation makes mistakes 😉.

I had to invest about 15 minutes of additional editing per episode to get the same result as with Descript. Which is ok when you are only casually podcasting.

So to get my speaker labels I did the diarisation manually and every time the speaker changes I added an empty line. The transcripts look like this:

I then fed the diarised output to chatGPT with a prompt to add the speaker labels.

Note: This only works well if you have two speakers that naturally alternate.

The prompt looks like this:

This worked for me, although sometimes I got the modified transcript as a codesnippet 😋.

For generating further content I used chatGPT-3.5 which can handle the longer transcripts and works decently now.

The results where sometimes of mixed quality and I highly recommend to proofread them and give them a personal touch.

But the speedup at this stage for creating a podcast description, a LinkedIn post and a social messenger post, was fairly impressive. For the five episodes I needed about two hours of work to get the quality I wanted.

Last time this took me five hours which is an impressive speedup 🥳.

In the next section I will share my prompts and the reasoning behind it. I didn’t do much prompt engineering to be honest. Just some basic things I picked up in the last weeks. If someone wants to improve on them let me know what worked for you.