Smart Playlists - duplicate track removal = smart pruning
Moderators: jiri, drakinite, Addon Administrators
Smart Playlists - duplicate track removal = smart pruning
In MM4, I had written a useful script that removes tracks with duplicate artists and titles from playlists. It worked by searching for matching artist and titles within the same playlist (with a significant amount of string processing for cleanup). The output was a manual playlist copy of the original (original=smart or manual) with the same name except for an asterisk at the end. For me, this was extremely useful for removing duplicate tracks from playlists that may be present on different albums - without having to delete one of the copies of the track, therefore, keeping all albums intact. The "pruned" playlist is what ends up being used for playback and synced to devices.
I would really like to duplicate this in MM5, however, it would have to be re-written from scratch as scripting has been totally changed. I have a programming background, but I really had to spend a lot of time writing the original MM4 script by "hacking" examples from others and I really don't have the time to re-do this (I tried a few months ago).
I think this could be a useful built-in feature of MM5, to add a "smart pruning" capability for playlists - the existing "remove duplicates", for manual playlists only doesn't really help - is this something that the developers would consider adding as a feature?
I would really like to duplicate this in MM5, however, it would have to be re-written from scratch as scripting has been totally changed. I have a programming background, but I really had to spend a lot of time writing the original MM4 script by "hacking" examples from others and I really don't have the time to re-do this (I tried a few months ago).
I think this could be a useful built-in feature of MM5, to add a "smart pruning" capability for playlists - the existing "remove duplicates", for manual playlists only doesn't really help - is this something that the developers would consider adding as a feature?
Re: Smart Playlists - duplicate track removal = smart pruning
Hi, adding script/addon like this should be quite easy.
In actions.js there is already an action for removing playlist duplicates by track.id, i.e. this code:
All that needs to be changed are the lines around hasher[track.id] like this:
And create new playlist like this and put the new list there:
So in your addon create actions_add.js and put following there:
And then put to to context menu of playlist by creating viewHandlers_add.js and puttin this code:
In actions.js there is already an action for removing playlist duplicates by track.id, i.e. this code:
Code: Select all
playlistRemoveDuplicates: {
title: function () {
return _('Remove duplicates');
},
icon: 'remove',
visible: function () {
if (!window.uitools.getCanEdit())
return false;
else {
var pl = resolveToValue(this.boundObject);
return (pl.parent != undefined && !pl.isAutoPlaylist); // to exclude root playlists node and auto-playlists
}
},
execute: function () {
var pl = resolveToValue(this.boundObject);
var list = pl.getTracklist();
list.whenLoaded().then(() => {
list.modifyAsync(() => {
var hasher = {};
listForEach(list, (track, idx) => {
if (hasher[track.id])
list.setSelected(idx, true); // duplicate
hasher[track.id] = true;
});
pl.removeSelectedTracksAsync(list);
});
});
}
},
Code: Select all
if (!hasher[track.title])
newList.add(track); // not a duplicate
hasher[track.title] = true;
Code: Select all
var newplaylist = app.playlists.root.newPlaylist();
newplaylist.name = pl.name + ' (filtered)';
newplaylist.commitAsync().then(function () {
newplaylist.addTracksAsync(newList);
});
So in your addon create actions_add.js and put following there:
Code: Select all
actions.playlistRemoveDuplicatesByTitle = {
title: function () {
return _('Remove duplicates by title');
},
hotkeyAble: true,
icon: 'remove',
visible: function () {
if (!window.uitools.getCanEdit())
return false;
else {
var pl = resolveToValue(this.boundObject);
return (pl.parent != undefined); // to exclude root playlists node
}
},
execute: function () {
var pl = resolveToValue(this.boundObject);
var list = pl.getTracklist();
var newList = app.utils.createTracklist();
list.whenLoaded().then(() => {
var hasher = {};
listForEach(list, (track, idx) => {
if (!hasher[track.title])
newList.add(track); // not a duplicate
hasher[track.title] = true;
});
var newplaylist = app.playlists.root.newPlaylist();
newplaylist.name = pl.name + ' (filtered)';
newplaylist.commitAsync().then(function () {
newplaylist.addTracksAsync(newList);
});
});
}
};
Code: Select all
nodeHandlers.playlist.menuAddons = nodeHandlers.playlist.menuAddons || [];
nodeHandlers.playlist.menuAddons.push(function (node) {
if (node && node.dataSource) {
return [{
action: bindAction(window.actions.playlistRemoveDuplicatesByTitle, () => {
return node.dataSource;
}),
order: 40,
grouporder: 10,
}];
};
return [];
});
Last edited by Ludek on Fri Sep 17, 2021 8:55 am, edited 3 times in total.
Re: Smart Playlists - duplicate track removal = smart pruning
The working script is here: https://www.dropbox.com/s/td1t3albp2jgr ... .mmip?dl=0
feel free to adjust/tune the code
feel free to adjust/tune the code
Re: Smart Playlists - duplicate track removal = smart pruning
Thanks! - I just saw this and downloaded
Update: Got it working well - although it is a different approach than my previous script (this new one uses hashing), it is working about 99% the same as my MM4 script for removing duplicate artists/titles - Thanks again!
Update: Got it working well - although it is a different approach than my previous script (this new one uses hashing), it is working about 99% the same as my MM4 script for removing duplicate artists/titles - Thanks again!
Re: Smart Playlists - duplicate track removal = smart pruning
I have been trying to enhance this script and ran into a problem - I want to create a copy of a smart playlist and have the script wait until that copy is ready before having the program proceed - I am trying to use playlist.createCopyAsync(); - I have a background in real-time C programming for embedded systems and am trying to catch up on javascript, promises, etc.
Given a playlist, how do I create the copy and have the script for it to be ready?
This doesn't seem to work (pl= the playlist to process and it is already available at this point)
var pl_sort = pl.createCopyAsync().then(() => {
// actions here
});
(Then I want to get the tracks, re-sort and use this new playlist as the one to run my previous script on so that it selects duplicates with the first track on the list being preferred)
Thanks in advance for any replies!
Given a playlist, how do I create the copy and have the script for it to be ready?
This doesn't seem to work (pl= the playlist to process and it is already available at this point)
var pl_sort = pl.createCopyAsync().then(() => {
// actions here
});
(Then I want to get the tracks, re-sort and use this new playlist as the one to run my previous script on so that it selects duplicates with the first track on the list being preferred)
Thanks in advance for any replies!
Re: Smart Playlists - duplicate track removal = smart pruning
Promises and async/await are a bit counter-intuitive until you get used to them. The variable type that's returned by pl.createCopyAsync() is actually a Promise, and the way to get the actual playlist copy is either via "await" or by taking a parameter from the ".then()" callback:
You don't have to do it in multiple lines, but hopefully this helps clear the confusion as to which variables are what:
here's an async/await way of doing the same thing:
https://developer.mozilla.org/en-US/doc ... c_function
You don't have to do it in multiple lines, but hopefully this helps clear the confusion as to which variables are what:
Code: Select all
pl2promise = pl.createCopyAsync();
pl2promise.then((pl2) => {
// do stuff with pl2
console.log(pl2.title);
console.log(pl2.asJSON);
})
Code: Select all
async function domystuff() {
var pl2 = await pl.createCopyAsync();
console.log(pl2.asJSON);
}
domystuff();
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
Re: Smart Playlists - duplicate track removal = smart pruning
Thanks - that worked - would you mind helping with the code to re-sort the playlist below?
...
// create copy of the playlist
var pl2promise = pl.createCopyAsync();
//---------------------------------------------------------------------------------
pl2promise.then((pl_sort) => {
pl_sort.name = pl.name + ' (sorted)';
pl_sort.parent = pl;
var tracks_sort = pl_sort.getTracklist();
tracks_sort.whenLoaded().then(() => {
tracks_sort.beginUpdate();
tracks_sort.setSortRule("Rating Z..A; Bitrate Z..A; Length Z..A;");
tracks_sort.endUpdate();
pl_sort.beginUpdate();
pl_sort.reorderAsync(tracks_sort);
pl_sort.endUpdate();
pl_sort.commitAsync();
The playlist is created but does not get sorted - still having difficulties with async programming in js - sometimes bad code will lock up MM5 -
...
// create copy of the playlist
var pl2promise = pl.createCopyAsync();
//---------------------------------------------------------------------------------
pl2promise.then((pl_sort) => {
pl_sort.name = pl.name + ' (sorted)';
pl_sort.parent = pl;
var tracks_sort = pl_sort.getTracklist();
tracks_sort.whenLoaded().then(() => {
tracks_sort.beginUpdate();
tracks_sort.setSortRule("Rating Z..A; Bitrate Z..A; Length Z..A;");
tracks_sort.endUpdate();
pl_sort.beginUpdate();
pl_sort.reorderAsync(tracks_sort);
pl_sort.endUpdate();
pl_sort.commitAsync();
The playlist is created but does not get sorted - still having difficulties with async programming in js - sometimes bad code will lock up MM5 -
Re: Smart Playlists - duplicate track removal = smart pruning
I did some digging and your question has led me to learn a lot about sorting and auto-playlists that I did not before. Several notes:
1. Documentation on tracklist sorting is terribly lacking. I've updated the documentation, but it'll be a bit before it's live on the site. Here's a copy of setAutoSortAsync and setSortRule that I've written:
setAutoSortAsync:
ASYNCHRONOUSLY sorts the list with the given sorting rule and updates the auto-sort rule.
To specify sort direction, append ' ASC' or ' DESC' to the tag name and separate them by semicolons.
Fields are NOT case sensitive (e.g. 'artist', 'Artist', and 'ArTiSt') are all valid, but sort direction IS case sensitive ('ASC' and 'DESC' are valid; 'asc' and 'desc' are not)
Valid examples:
list.setAutoSortAsync('title');
list.setAutoSortAsync('artist; title');
list.setAutoSortAsync('rating DESC; title ASC');
list.setAutoSortAsync('Rating DESC; TITLE;');
Invalid examples:
list.setAutoSortAsync('tagThatDoesNotExist');
list.setAutoSortAsync('rating desc; title asc');
setSortRule:
SYNCHRONOUSLY sorts the list with the given sorting rule and disables auto-sort.
To specify sort direction, append ' ASC' or ' DESC' to the tag name and separate them by semicolons.
Fields are NOT case sensitive (e.g. 'artist', 'Artist', and 'ArTiSt') are all valid, but sort direction IS case sensitive ('ASC' and 'DESC' are valid; 'asc' and 'desc' are not)
Valid examples:
list.setSortRule('title');
list.setSortRule('artist; title');
list.setSortRule('rating DESC; title ASC');
list.setSortRule('Rating DESC; TITLE;');
Invalid examples:
list.setSortRule('tagThatDoesNotExist');
list.setSortRule('rating desc; title asc');
2. You don't need to do beginUpdate() and endUpdate() in this case. It is useful when we have UI controls who have listened to certain events from their dataSources. BUT since this is a brand-new clone, and you have not listened to events (e.g. app.listen() or localListen()), it'll have no impact (description of beginUpdate: Lock object to update state. Events are not called when in update state.)
3. You'll actually need different code for auto-playlists and regular playlists. You were using reorderAsync() correctly for a regular playlist, but auto-playlists use a more complex sorting method and reorderAsync() does not work for them. (I've also updated the docs to note this)
Auto Playlists use a QueryData object to handle their sorting. You can check searchEditor.js and playlistHeader.js for how these QueryData objects are created and updated. Also, the property isAutoPlaylist can be read to check if a given playlist is an auto-playlist AND updated in case you want to switch its type.
4. It's better to use tracks_sort.setAutoSortAsync() to avoid blocking the main UI thread.
5. Because of all the async functions required, it'll be much easier to use an async function and use await each in order. (Otherwise, you'd be doing a lot of .then()s and lots of annoying indentation, a.k.a. "callback hell")
All together:
1. Documentation on tracklist sorting is terribly lacking. I've updated the documentation, but it'll be a bit before it's live on the site. Here's a copy of setAutoSortAsync and setSortRule that I've written:
setAutoSortAsync:
ASYNCHRONOUSLY sorts the list with the given sorting rule and updates the auto-sort rule.
To specify sort direction, append ' ASC' or ' DESC' to the tag name and separate them by semicolons.
Fields are NOT case sensitive (e.g. 'artist', 'Artist', and 'ArTiSt') are all valid, but sort direction IS case sensitive ('ASC' and 'DESC' are valid; 'asc' and 'desc' are not)
Valid examples:
list.setAutoSortAsync('title');
list.setAutoSortAsync('artist; title');
list.setAutoSortAsync('rating DESC; title ASC');
list.setAutoSortAsync('Rating DESC; TITLE;');
Invalid examples:
list.setAutoSortAsync('tagThatDoesNotExist');
list.setAutoSortAsync('rating desc; title asc');
setSortRule:
SYNCHRONOUSLY sorts the list with the given sorting rule and disables auto-sort.
To specify sort direction, append ' ASC' or ' DESC' to the tag name and separate them by semicolons.
Fields are NOT case sensitive (e.g. 'artist', 'Artist', and 'ArTiSt') are all valid, but sort direction IS case sensitive ('ASC' and 'DESC' are valid; 'asc' and 'desc' are not)
Valid examples:
list.setSortRule('title');
list.setSortRule('artist; title');
list.setSortRule('rating DESC; title ASC');
list.setSortRule('Rating DESC; TITLE;');
Invalid examples:
list.setSortRule('tagThatDoesNotExist');
list.setSortRule('rating desc; title asc');
2. You don't need to do beginUpdate() and endUpdate() in this case. It is useful when we have UI controls who have listened to certain events from their dataSources. BUT since this is a brand-new clone, and you have not listened to events (e.g. app.listen() or localListen()), it'll have no impact (description of beginUpdate: Lock object to update state. Events are not called when in update state.)
3. You'll actually need different code for auto-playlists and regular playlists. You were using reorderAsync() correctly for a regular playlist, but auto-playlists use a more complex sorting method and reorderAsync() does not work for them. (I've also updated the docs to note this)
Auto Playlists use a QueryData object to handle their sorting. You can check searchEditor.js and playlistHeader.js for how these QueryData objects are created and updated. Also, the property isAutoPlaylist can be read to check if a given playlist is an auto-playlist AND updated in case you want to switch its type.
4. It's better to use tracks_sort.setAutoSortAsync() to avoid blocking the main UI thread.
5. Because of all the async functions required, it'll be much easier to use an async function and use await each in order. (Otherwise, you'd be doing a lot of .then()s and lots of annoying indentation, a.k.a. "callback hell")
All together:
Code: Select all
pl.createCopyAsync().then(async (pl_sort) => {
pl_sort.name = pl.name + ' (sorted)';
pl_sort.parent = pl;
// Do QueryData shenanigans if it's an auto-playlist
if (pl_sort.isAutoPlaylist) {
let queryData = await app.db.getQueryData({ category: 'empty' }); // Create a new QueryData object
queryData.loadFromString(pl_sort.queryData); // Transfer all its properties from the playlist (essentially creating a copy)
queryData.setSortOrders([ // Set its sort order: this time it's an array of objects insteada of a string
{
name: 'Rating',
ascending: false,
},
{
name: 'Bitrate',
ascending: false,
},
{
name: 'Length',
ascending: false,
},
]);
pl_sort.queryData = qd.saveToString(); // Update the playlist's queryData
// (Note: TPlaylist.queryData isn't actually stored as a string. Internally, it uses saveToString() as a getter and loadFromString() as a setter.)
pl_sort.commitAsync(); // Commit the playlist
pl_sort.notifyChanged('tracklist'); // to live update tracks -- is listened e.g. by viewHandlers.tracklistBase.onShow
}
// otherwise, use the tracklist sort method
else {
let tracks_sort = pl_sort.getTracklist();
await tracks_sort.whenLoaded();
await tracks_sort.setAutoSortAsync("Rating DESC; Bitrate DESC; Length DESC;");
await pl_sort.reorderAsync(tracks_sort);
pl_sort.commitAsync();
}
});
Last note is unrelated to playlists and tracklists: MM5 will often lock up and crash if you do something wrong with a native Delphi object or method (All of the objects we've been messing with here are native MM objects). When in doubt, check the documentation, and if the documentation isn't clear, don't hesitate to ask for us to clarify (and update the documentation to be more clear).
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
Re: Smart Playlists - duplicate track removal = smart pruning
Thank you for the in-depth information - after some major re-writing, the script is now working better than its MM4 counterpart
Re: Smart Playlists - duplicate track removal = smart pruning
Excellent to hear!
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
Re: Smart Playlists - duplicate track removal = smart pruning
Wasn't that purpose of MM5 as it means that years of developing was worth.
Best regards,
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
How to attach PICTURE/SCREENSHOTS to forum posts
Peke
MediaMonkey Team lead QA/Tech Support guru
Admin of Free MediaMonkey addon Site HappyMonkeying
How to attach PICTURE/SCREENSHOTS to forum posts
-
- Posts: 2
- Joined: Wed Dec 27, 2023 11:33 pm
Re: Smart Playlists - duplicate track removal = smart pruning
Is this script available for download? Thanks!