Skip to main content

Copy only directories with .mp3 files with rsync [Resolved]

I'm trying to copy only folders with .mp3 files in them to another device. And only the .mp3s in them. The structure looks like this:

Band/Year - Album/*.flac
Band/Year - Album/*.txt
Band/Year - Album/*.mp3
Band/Year - Album/covers
Band/Year - Album/covers/*.jpg

I've already read a shitton of manuals and forums but can't resolve my problem. No matter what I do, no matter how I'm trying to formulate what I want to include in copying, since it's excluded - it's gone even

--exclude=* --include=*

won't copy anything. I tested it on another directory, even without any, dammit.

EDIT: List of what I tried to no avail:

rsync -rnv --exclude=** --include=*/ --include=*/*/ --include=*/*/*/ --include=*/*/*.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=** --include=*/ --include=*/*/ --include=*/*/*/ --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=** --include=*/ --include=*/* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=*/*/* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude=*/*/*.* --include=**.mp3 "./Be'lakor" /media/moorindal/WALKMAN/MUSIC
rsync -rnv --exclude='*/*/*.*' --include='*/*/*.mp3' "./Be'lakor" /media/moorindal/WALKMAN/MUSIC

Question Credit: Yuri Drobkov
Question Reference
Asked April 5, 2019
Posted Under: Unix Linux
3 Answers

First, why --exclude=* --include=* doesn't work: because the first matching rule applies. * matches everything, so everything is excluded, and rsync doesn't even try the second rule for any file. See Rsync filter: copying one pattern only for a guide to rsync filters.

To include .mp3 files, you need

rsync -a -m --include='*.mp3' --include='*/' --exclude='*'

i.e. include MP3, include all directories (if a directory isn't included then none of the files inside it are included) and exclude everything else. The -m option (a.k.a. --prune-empty-dirs) makes rsync skip directories that don't contain any files to be copied. But that won't include other files in the same directory as the .mp3 file. For that, you need some help from the shell or other tools.

In zsh, you can match .mp3 files in subdirectories with **/*.mp3, and then use a history modifier as a glob qualifier to transform the result into the list of directories containing .mp3 files.

rsync -a **/*.mp3(:h) /destination

If you have too many directories (or more precisely, if the combined length of their names is too large), this may break the command line limit. Removing duplicates might help:

typeset -aU directories
rsync -a $directories /destination

This doesn't eliminate the risk that the command is too long, it merely reduces it. To eliminate the risk, use zargs to run rsync in batches.

autoload -U zargs
typeset -aU directories
do_rsync () { rsync -a $@ /destination; }
zargs --max-chars=$(($(get_conf ARG_MAX) - 500)) -- $directories -- do_rsync

credit: Community
Answered April 5, 2019

You're trying to do too much with rsync. The first thing to do is to produce the list of all the MP3 files.

find Music -type f -iname '*.mp3'

Then convert the list of files into a list of the directories that contain the files.

sed 's:/[^/]*$::'

Eliminate any duplication of directories in the list.

sort -u

Now you have the list of directories you need. Feed them to rsync one at a time. Use --relative to keep the directory tree intact at the destination.

while IFS= read -r dir; do rsync --archive --relative "$dir" /mnt; done

Altogether as a single pipeline:

find Music -type f -iname '*.mp3' |
   sed 's:/[^/]*$::' |
   sort -u |
   while IFS= read -r dir; do rsync --archive --relative "$dir" /mnt; done

credit: Gilles
Answered April 5, 2019
Your Answer