nevernude: Automatically cut out NSFW nudity from videos using Machine Box + ffmpeg

How will our solution work?

We have to do four simple steps in order to achieve our goal:

1. Send the video to Videobox+Nudebox to get a list of time spans that contain nudity
2. Use that information to generate a list of time spans that therefore do not contain nudity
3. Use ffmpeg to break the original video into appropriate segments
4. Use ffmpeg to stitch the segments back into a new video file

Machine learning is never 100% accurate, so videos automatically edited using this technique should be double-checked before publication.

Videobox+Nudebox

The easiest way to spin up Videobox and Nudebox on your local machine is to use Docker compose (learn more on Docker’s website).

Create a folder called nevernude, and insert the following docker-compose.yml file:

version: ‘3’
services:
  nudebox1:
    image: machinebox/nudebox
    environment:
      - MB_KEY=${MB_KEY}
    ports:
      - "8081:8080"
    videobox:
      image: machinebox/videobox
      environment:
        - MB_KEY=${MB_KEY}
        - MB_VIDEOBOX_NUDEBOX_ADDR=https://nudebox1:8080
      ports:
        - "8080:8080"

You’ll need to have Docker installed and the the MB_KEY variable set.

In a terminal, run docker-compose up which will spin up two Docker containers, one for Nudebox and one for Videobox.

A little program to process video

In this article we are going to look at a solution using Go, but you can use any language you like (or even a bash script) — after all, Machine Box provides simple RESTful JSON APIs that are easy to consume in any language.

The complete source code for the nevernude tool is available to browse directly. This blog post omits lots of boilerplate stuff, including creating temporary directories etc. to focus on the important things. See the source code for a complete picture.

Analyse the video with Videobox

We are going to use the Machine Box Go SDK to help us make requests (but they’re just HTTP requests, so you can use curl if you like).

Create a Videobox client and use it to process the source video file:

// TODO: load the source file into src

vb := videobox.New("https://localhost:8080")
video, err := vb.Check(src, opts)
if err != nil {
    return errors.Wrap(err, "check video")
}
results, video, err := waitForVideoboxResults(vb, video.ID)
if err != nil {
    return errors.Wrap(err, "waiting for results")
}

• videobox.New creates a new videobox.Client that provides helpers for accessing the services that are running locally in the Docker containers we spun up
• vb.Check sends the video file to Videobox for processing

The waitForVideoboxResults function periodically checks the status of the video (with the specified video.ID) before getting and returning the results:

Once the waitForVideoboxResults function returns, we’ll have the videobox.VideoAnalysis object which contains the nudity information.

• The function also deletes the results from Videobox, freeing resources

Create a collection of non-nude segments

Next we need to use the nudity instances to create a list of time ranges that do not contain nudity.

For example, if a ten second video contains nudity from 4s–7s, we would expect two non-nude segments, 0s–3s and 8s-10s — thus omitting the nude bits.

Here’s the code:

type rangeMS struct {
    Start, End int
}

var keepranges []rangeMS
s := 0
for _, nudity := range results.Nudebox.Nudity {
    for _, instance := range nudity.Instances {
        r := rangeMS{
            Start: s,
            End:   instance.StartMS,
        }
        s = instance.EndMS
        keepranges = append(keepranges, r)
    }
}

keepranges = append(keepranges, rangeMS{
    Start: s,
    End:   video.MillisecondsComplete,
})

In ffmpeg, we can use the following command to extract segments based on these ranges:

ffmpeg -i input.mp4 -ss {start} -t {duration} segment1.mp4

{start} is the number of seconds to seek in the original video (the start of the segment), and {duration} is the length of the segment in seconds. segment1.mp4 is the filename of the segment to create.

We’ll also create a text file that ffmpeg understands that lists each segment — which we’ll use later to stitch them back together. The file will follow this format:

file 'segment1.mp4'
file 'segment2.mp4'
file 'segment3.mp4'
etc

Since we don’t know how many segments there are going to be, we’ll use code to generate the ffmpeg arguments and segment list file.

The ffmpegargs variable is a string of arguments that we can pass into the command when we execute it.

ffmpegargs := []string{
    "-i", inFile,
}

listFileName := filepath.Join(tmpdir, “segments.txt”)
lf, err := os.Create(listFileName)
if err != nil {
    return errors.Wrap(err, “create list file”)
}
defer lf.Close()

for i, r := range keepranges {
    start := strconv.Itoa(r.Start / 1000)
    duration := strconv.Itoa((r.End - r.Start) / 1000)
    segmentFile := fmt.Sprintf(“%04d_%s-%s%s”, i, start, start+duration, ext)
    segment := filepath.Join(tmpdir, segmentFile)
    _, err := io.WriteString(lf, “file '“+segmentFile+”'\n”)
    if err != nil {
        return errors.Wrap(err, “writing to list file”)
    }
    ffmpegargs = append(ffmpegargs, []string{
        “-ss”, start,
        “-t”,  duration,
        segment,
    }...)
}

• We are dividing the Start and End values by 1000 because they’re in milliseconds, but ffmpeg wants seconds
• The %04d_%s-%s%s string just creates a nice filename (made up of the index, start and end times) which is helpful for debugging
• The tmpdir variable is a temporary folder where the tool can store the segments and list file — it can be deleted once the final video is completed

Finally, we’ll use exec.Command from Go’s standard library to execute ffmpeg:

out, err := exec.Command(“ffmpeg”, ffmpegargs...).CombinedOutput()
if err != nil {
    return errors.Wrap(err, “ffpmeg: “+string(out))
}

If we ran this program, we’d end up with a folder that contained the segments and the list file:

Stitch segments together into a new video

Finally, we must use ffmpeg to stitch all of the segments listed in our segments.txt file into a new video:

ffmpeg -f concat -safe 0 -i segments.txt -c copy output.mp4

So in Go code this would be:

ffmpegargs = []string{
    “-f”, “concat”, “-safe”, “0”,
    “-i”, listFileName, “-c”, “copy”, output,
}

out, err = exec.Command(“ffmpeg”, ffmpegargs...).CombinedOutput()
if err != nil {
    return errors.Wrap(err, “ffpmeg: “+string(out))
}

Running this program will take a source video, and create a new one with the nudity omitted.

To make this solution more robust, you might decide to add a little bit of time either side of detected nudity, just to be sure to cut all of it out. That can be your homework.

Customisation

Videobox lets you control the threshold (Nudebox confidence value) before frames are considered to contain nudity. Just pass the nudeboxThresholdvalue (a number between 0 and 1, where zero is the most strict) or use the NudeboxThreshold option in the Go SDK.

You can also specify additional options to control Videobox, including how many frames are processed, etc.

Conclusion

We saw how easy it was to mashup Machine Box and ffmpeg to process a video to remove sections of detected nudity.

Head over to the nevernude project on Github to see the complete code, and even run it on your own videos.