beet convert only files that have not been converted to the target format yet?

[j9ac9k]

Hello beets community,

I'm new to the project, but with the help of the documentation, I have my beets config file fairly well tuned, but there are a few pain points I still have that I'm hoping I can get help with.

The relevant sections of my config file are as suhc...

paths:
    default: $format/Artists/$albumartist/$album%aunique{}/$track - $title
    singleton: $format/Non-Album/$artist - $title
    comp: $format/Compilations/$album%aunique{}/$track - $title
    albumtype:soundtrack: $format/Soundtracks/$album/$track $title

convert:
    auto: no
    ffmpeg: /opt/homebrew/bin/ffmpeg
    max_bitrate: 320
    threads: 1
    dest: "~/Music/Manual"
    formats:
        mp3:
            command: ~/Music/Manual/beet_extras/lame_gapless.sh $source $dest
        alac:
            command: ffmpeg -i $source -y -vn -acodec alac $dest
            extension: m4a
        aac:
            command: ffmpeg -i $source -y -vn -acodec aac -aq 1 $dest
            extension: m4a
        flac:
            command: ffmpeg -i $source -y -vn -acodec flac $dest

The majority of my music collection is in flac format, however, I still use iPod's (what can I say, I'm nostalgic and the touch-user interface cannot be beat), so I effectively have a duplicate library in alac format, and let Apple Music automatically convert to aac when I sync to my iPod.

I have two problems when I convert, when I run two things happen that I wish I could avoid

beet convert format:flac --format alac --dest ./ALAC

1. the FLAC directory is preserved so my converted files are ./ALAC/FLAC/Artist/Album/..., would be good to know if there was a way to have the output go to ./ALAC/Artist/Album/....
2. beets converts content that has already been converted; I just want it to convert stuff that has yet to be converted.

Any input would be greatly appreciated, thanks for creating and maintaining such an awesome tool!

EDIT: I decided for point 2 that I could likely do some fancy query syntax, but I don't seem to have gotten any luck:

$ beet ls format:flac | wc -l
2950
$ beet ls format:alac | wc -l
2215
$ beet ls format:flac ^format:alac | wc -l
2950

[j9ac9k]

In case anyone comes here from google, the way I worked around problem number two was some really lame python to get a list of artists, albums, and print out the command needed to run to convert the files. Still a lot of copy/paste, but it's fine.

import subprocess

flac_album_command = [
	"beet",
	"list",
	"-a",
	"format:flac"
]
flac_album_result = subprocess.run(
	flac_album_command,
	capture_output=True,
	text=True
)
flac_albums = set(flac_album_result.stdout.split("\n"))

alac_album_command = [
	"beet",
	"list",
	"-a",
	"format:alac"
]
alac_album_result = subprocess.run(
	alac_album_command,
	capture_output=True,
	text=True
)
alac_albums = set(alac_album_result.stdout.split("\n"))
artists_albums_to_sync = flac_albums - alac_albums
for entry in artists_albums_to_sync:
	artist, _, album = entry.partition(" - ")
	print(f"beet convert artist:'{artist}' album:'{album}' -y format:flac  --format alac --dest ./ALAC")

[jaawerth]

@j9ac9k

1. Looks like the convert plugin needs something like a dest_format template field - based on the code I'm guessing overriding the default format will make it confuse source/dest and throw a fit.
   I decided to poke at it since I've been looking at the plugin API lately anyway with thoughts on writing one:

   This quick hack on the convert plugin seems to work - or at least, it prints the right commands with convert -a -p album:matchonealbum when I've put $dest_format in convert's path configs, but I didn't actually try to run it for real since my library isn't structured that way, so 🤷.

   Anwyay, if someone wants to take the time to add tests and such to this and work it into a PR, feel free - I'm not sure if I'll have the time.

   diff --git a/beetsplug/convert.py b/beetsplug/convert.py
   index e72f8c75a..0f29b8832 100644
   --- a/beetsplug/convert.py
   +++ b/beetsplug/convert.py
   @@ -162,6 +162,7 @@ class ConvertPlugin(BeetsPlugin):
   	 }
        )
        self.early_import_stages = [self.auto_convert, self.auto_convert_keep]
   +        self.template_fields['dest_format'] = self._tmpl_dest_format
    
        self.register_listener("import_task_files", self._cleanup)
    
   @@ -770,3 +771,6 @@ class ConvertPlugin(BeetsPlugin):
        ]
        pipe = util.pipeline.Pipeline([iter(items), convert])
        pipe.run_parallel()
   +
   +    def _tmpl_dest_format(self, _tgt):
   +        return self.config["dest_format"]

2. That's weird, what version are you on? For me (on 2.0.0, but I feel like it's done that I first started using beets a couple years ago - since at least as far back as 1.6.0), it automatically skips any tracks where the destination already exists.

   Assuming you're not on an ancient version, I'm starting to think these problems might be related; maybe you hit a weird edge case.. just a shot in the dark, but I'm wondering if that weird /FLAC/ALAC/ is causing inconsistent results in where it thinks the path is. I'd try temporarily configuring paths on the convert plugin level without $format and see if it skips properly then. At least that way you'll know it's one problem rather than two 😂

[j9ac9k]

1. I'll give your diff a shot later; but it wouldn't surprise me that it's a very quick correct if there is already a dest_format field that can be populated.
2. I'm using the current version of beets (2.5), but I suspect this is an issue in part due to the weird paths due to the first issue; I wouldn't be surprised if this issue resolves itself after getting the destination path sorted out.

Thanks for chiming in!

[j9ac9k]

@j9ac9k

1. Looks like the `convert` plugin needs something like a `dest_format` template field - based on the code I'm guessing overriding the default `format` will make it confuse source/dest and throw a fit.
   I decided to poke at it since I've been looking at the plugin API lately anyway with thoughts on writing one:
   This quick hack on the convert plugin seems to work - or at least, it prints the right commands with `convert -a -p album:matchonealbum` when I've put `$dest_format` in convert's path configs, but I didn't actually try to run it for real since my library isn't structured that way, so 🤷.
   Anwyay, if someone wants to  take the time to add tests and such to this and work it into a PR, feel free - I'm not sure if I'll have the time.
   ```diff
   diff --git a/beetsplug/convert.py b/beetsplug/convert.py
   index e72f8c75a..0f29b8832 100644
   --- a/beetsplug/convert.py
   +++ b/beetsplug/convert.py
   @@ -162,6 +162,7 @@ class ConvertPlugin(BeetsPlugin):
   	 }
        )
        self.early_import_stages = [self.auto_convert, self.auto_convert_keep]
   +        self.template_fields['dest_format'] = self._tmpl_dest_format
    
        self.register_listener("import_task_files", self._cleanup)
    
   @@ -770,3 +771,6 @@ class ConvertPlugin(BeetsPlugin):
        ]
        pipe = util.pipeline.Pipeline([iter(items), convert])
        pipe.run_parallel()
   +
   +    def _tmpl_dest_format(self, _tgt):
   +        return self.config["dest_format"]
   ```

2. That's weird, what version are you on? For me (on 2.0.0, but I feel like it's done that I first started using beets a couple years ago - since at least as far back as 1.6.0), it automatically skips any tracks where the destination already exists.
   Assuming you're not on an ancient version, I'm starting to think these problems might be related; maybe you hit a weird edge case..  just a shot in the dark, but I'm wondering if that weird `/FLAC/ALAC/` is causing inconsistent results in where it thinks the path is. I'd try temporarily configuring `paths` on the convert plugin level _without_ `$format` and see if it skips properly then. At least that way you'll know it's _one_ problem rather than two 😂

I applied the diff to ./lib/python3.13/site-packages/beetsplug/convert.py (had to undo one level of indentation) but I am still getting the same behavior.

$ beet convert -a -p album:<album> --format alac --dest ./

convert: ffmpeg -i /Users/ogi/Music/Manual/MP3/Soundtracks/<album>/<track>.mp3 -y -vn -acodec alac /Users/ogi/Music/Manual/MP3/Soundtracks/<album>)/<track>.m4a

I'll spend some time editing convert.py and see if I can get it working...

[j9ac9k]

Ok, I fixed both issues in one go with making one change to convert.py. I may submit as a PR, but I'm not sure this is the "right" way to fix it:

    def convert_item(
        self,
        dest_dir,
        keep_new,
        path_formats,
        fmt,
        pretend=False,
        link=False,
        hardlink=False,
    ):
        """A pipeline thread that converts `Item` objects from a
        library.
        """
        command, ext = get_format(fmt)
        item, original, converted = None, None, None
        while True:
            item = yield (item, original, converted)
            item.format = fmt.upper()  # <-- add this line here
            dest = item.destination(basedir=dest_dir, path_formats=path_formats)

Issue stems from the item.destination giving the wrong destination back because it thinks the item.format is not updated as part of the convert process, we only change the file extension a few lines further down.

[j9ac9k]

Created #6105