We need a 'Sync-Cache' or a 'Shadow-Library' [#7636]

[wolfzell]

While portable devices are getting more and more memory nowadays and libraries are getting more and more lossless, the syncing process is starting to get a real problem. My 32 GB iPod Touch is needing almost 2 days to get synced from my Flac library... 2 days in which I can not use the portable device. This is very user-unfriendly.

So a solution is needed.

For the purpose of syncing portable devices, multiple libraries do not seem like a good way to go as this means more work by yourself when managing them and no way of getting tag changes (i.e. ratings) from the portable devices back to the original library. Instead a much better solution could be something like a "Sync-Cache".

A "Sync-Cache" means:

MM should be able to point to a "Sync-Cache"-directory (beneath the original library), where the same tracks are stored, already converted to the format of the portable device, so you won't need to convert them over and over again on-the-fly, when syncing. Instead for syncing purposes MM should simply get the already converted files from this alternative "Sync-Cache"-directory.

This means: You can manage tracks and playlists and everything via the single original library in MM using every kind of file format you fancy. But when syncing, MM is retrieving already once converted files from the "Sync-Cache", while only updating tag information for these files from the original database in this process. If the files do not exist in the "Sync-Cache", it should still convert them on the fly and also store the conversion result in the "Sync-Cache" for future syncs which won't need the slow on-the-fly-conversion for these songs again then.

This would avoid converting the same songs over and over again when syncing.

I guess this could best be implemented in the rules section for "auto-converting" as an additional option for the already existing conversion options. Could look like this: "Incompatible file formats: Convert to XX using Sync-Cache in directory YY.

On top of that it would be nice to have an option to have MM generate this "Sync-Cache" at once for the whole library as a "Shadow-Library" and some way to keep it in sync. Then you wouldn't need any on-the-fly-conversion while syncing portable devices anymore.

Does this sound like a good idea or did I miss something which would make this impossible or unusable?

bye
Wolfgang

[theshawn]

Please, please, please, pretty please could someone make this possible. If I knew how to go about such a thing I would have built a program or add-on that could do this years ago. I like the way you call it a 'Sync-cache', that's what I have been envisioning for years! I thought the on-the-fly would be good enough, but it seems that MM re-converts each time it re-syncs (thus making for the described long syncs)

[MoDementia]

I'm running a Sync-Cache now as my main library for the same reason, unfortunately my original library is missing out on recent tag corrections

[Teknojnky]

This is a pretty good idea I think, as long as its configurable (where to store the cache and how big it should be).

I wonder if the virtual cd function can act as this functionality.. it depends on if the already converted virtual cd files would be used instead of the source file.

[wolfzell]

I also thought that the virtual cd or the preview function could evolve into something like my idea, but as I fear some people might miss the original functions, I thought of my solution above as it would not need much change in other areas of MM.

All that would be needed would be a configuration page for the paths and filenames where the "cache" should be stored and a new option for the conversion options for portable devices. Seems like a reasonable amount of changes for the whole lot of advantages it would provide.

The buildling of a whole shadow-library and keeping it in sync with the original then could be done via a script I guess.

bye
Wolfgang

[wolfzell]

I'm running a Sync-Cache now as my main library for the same reason, unfortunately my original library is missing out on recent tag corrections

That is the problem if you start managing 2 libraries (one lossless and one for portable devices). You wil have *a lot* of additional managing to do to keep your tags and playlists in sync between the 2 libraries. Actually I firmly believe that with a reasonable library size (5000 track+) it simply isn't a good solution anymore.

That is the reason why I came up with the idea above. Would solve all those problems with a reasonable complexity and does not need changes in MMs database format or basic program functions. I can't think of any easier way to solve the problem.

bye
Wolfgang

[wolfzell]

This is a pretty good idea I think, as long as its configurable (where to store the cache and how big it should be).

Sure. I didn't want to flesh out the basic idea in too much detail. I am not sure about the size limit (hard disk space is so cheap...), but the place and the filenames should be configurable.

bye
Wolfgang

[Al_G]

Does this sound like a good idea or did I miss something which would make this impossible or unusable?

bye
Wolfgang

It's a very good ideal and I'm sure you've suggested it before or it's been discussed in which you participated). Only possible drawback for users would be lack of available drive space for the cache copy. Even with flash drive players it would be a real time saver.

[gab]

I am trying to accomplish this with the autosync feature and the usb plug in. In essence, I have an MM lossless library which I then auto-sync to a folder on my hard drive (thereby creating a lossy library). When I edit tags on the lossless files I run auto-sync and it updates the tags of the lossy library (without needing to convert the songs in question). Similarly if I rip a new CD, I run the auto-sync to create a lossy version of the file.

My only problem right now is that I'm having some difficulty with the auto-sync feature (which I've described in another post).

[SmoochiesBoss]

I agree that this would be a great feature. Though the forward compatibility of MM and Apple products seems to be in doubt, I still want to be able to use MM as my PC-based flac manager. I'll concede my iPhone and Shuffle to Apple and use iTunes only to sync files onto those devices.

I just don't want to have to manually convert everything to mp3/mp4 and then manually keep all of those files and tags in sync with my flacs. MM should do it for me seamlessly in the background, just like it monitors specific directories for new, changed, and deleted files. As storage gets cheaper and cheaper, this option will become accessible to more and more people.

[VRN]

I certainly second the motion for a shadow or mirror sync function to allow any changes in a lossless master copy to be automatically reflected in a compressed (but otherwise identical) format.

See my questions/suggestions here: http://www.mediamonkey.com/forum/viewto ... 12&t=31587

In any event, I wonder why it is possible to sync to specific devices such as iPods when it is not possible to sync to simple generic devices such as a path on a harddrive or a NAS etc.

Best regards,

>V<

[Lowlander]

You can use the USBMass plugin to synchronize to a drive letter (set the drive letter on the Device Configuration tab). If you use a NAS or network location you will need to map it to a drive letter, but otherwise this works well.

[VRN]

Really!? That is great news. Thank you, I'll try it out.

Best regards,

>V<

[kdm]

Thanks for the tip Lowlander. This was exactly what I was looking for to sync my FLAC libray to an MP3 library on my Eee PC across a network (see thread referred to by VRN a few posts back).

[DWSR]

I personally think this idea lacks vision, or at least more details.

How are you going to address the problem of the second library growing and taking up a large amount of disk space?
Are you going to cache as soon as tracks are marked to sync, or are you going to just cache a copy as the tracks get converted during a sync process?
Will the MM library database be able to handle the addition of new fields to it?

The big issue for me is #1, however. I believe that a multi-threaded encoding process will drastically reduce the amount of time that it will take to sync PAPs, while providing no increased disk usage.