Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

Post a reply

Smilies
:D :) :( :o :-? 8) :lol: :x :P :oops: :cry: :evil: :roll: :wink:

BBCode is ON
[img] is ON
[flash] is OFF
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Thu Jan 14, 2021 6:16 pm

ZvezdanD wrote: Thu Jan 14, 2021 3:53 pm Well, in such case I suppose it would be better to use Folder instead of the Path field. You could try the predefined mask "Drives & Folders (string part)": replace Path with Folder and, if you don't want Sort/Stat, you could replace Split Mode:String Part with Split Mode:Single Part to get faster nodes.
You and your addon never cease to amaze. I'm sure you had probably thought of this before.

I just tried that predefined mask, both as default and with your change suggestions.

Default: I can only make it navigate down 5 levels before MediaMonkey fills the RAM again, and it freezes/crashes. Similar to my original post. Perhaps it is the the size of my library that is causing this issue across masks that dig deeper than ~5 levels.

Changed String Part to Single Part + Changed Path to Folder: It is slower, but very interestingly the RAM goes up a bit while processing, and then falls back down again to ~80mb. I haven't seen this behavior anywhere else using your addon. It seems to be the change to "Single Part", as I changed one node back to "Split Part" and the RAM fills up substantially more. However with "Single Part" I can use Folder or Path and MediaMonkey RAM usage always falls back after it finishes processing the currently selected node. This is the same for populating as either a Magic Node, or as a Playlist. I can definitely use this.

One caveat is that I've discovered this will not work with folders that contain brackets [ ]. Nodes with a [Folder] named in brackets fails to expand or populate its contents. Is there a workaround for this, e.g. an escape character? If not, I can rename all my folders, it will just be a project to take on.

Edit: Working mask as follows. I modified the one you mentioned so that I can specify a certain folder as the root (using Exclusive Right Of), then using only Split Part subnodes for the subfolders.

Code: Select all

Whole Library|Icon:Top level|Child of:Playlists|Position:Child\<Path|Unknown:No|Exclusive right of:Downloads\|Right until:\>\<Path|Split by:\|Split Mode:Single Part|Split part:5>\<Folder|Split by:\|Split Mode:Single Part|Split part:6>\<Folder|Split by:\|Split Mode:Single Part|Split part:7>

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by ZvezdanD » Thu Jan 14, 2021 3:53 pm

benfucius wrote: Thu Jan 14, 2021 1:54 pm I want to restrict the nodes to only Folders, to exclude Files as nodes. Any idea how to use any functions like Split, etc to remedy this?
Well, in such case I suppose it would be better to use Folder instead of the Path field. You could try the predefined mask "Drives & Folders (string part)": replace Path with Folder and, if you don't want Sort/Stat, you could replace Split Mode:String Part with Split Mode:Single Part to get faster nodes.

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Thu Jan 14, 2021 1:54 pm

Separate related topic:

I'm learning more about Split, reading your documentation on it (from the older MN version page) today. I am wondering if there may be some ways to use Split instead of needing symbols in my folders for Right Of uses (+,~,=,etc)

Edit: Read and tested, think I've found a way to get rid of the symbols in my folder names.

Code: Select all

<Path|Unknown:No|Exclusive right of:(path specific)|Split by:\|Split Mode:String Part|Split part:1,2,3,4,5 (depending on node level)
This seems to work similarly to having hard-coded symbols to differentiate the folder levels, but is more elegant in that it simply shows the folder that corresponds to the level depth. i.e. it mirrors the folder structure as nodes.

Use case: A folder 3 levels deep is where I add new "unorganized" music. I set the Exclusive right of:NewMusic, and then subnode(s) as copies of this mask, but increasing the Split part: up by one increment, down through to the end of the folder levels. This would effectively "pull out" this folder tree as a MN or Playlist unto itself, making it easier to access directly without needing to first navigate down 3 levels to reach "NewMusic", and also easier for syncing to the phone.

Issue/Question: If in a folder there are two folders, one has songs directly within it, and the other has more folders within it, using the above mask the nodes build out to include the song names in the nodes for the more shallow folders. This is if using either Path or Folder. Example below:

{Playlists} ~
|----[Rock] #
|--------All for Metal
|------------Song 1
|------------Song 2
|----Classical Essentials
|--------Song 3
|--------Song 4

Turns out like this using my above mask:
Image

I want to restrict the nodes to only Folders, to exclude Files as nodes. Any idea how to use any functions like Split, etc to remedy this?

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Thu Jan 14, 2021 10:56 am

ZvezdanD wrote: Thu Jan 14, 2021 4:43 am Thanks for the detailed report. My suggestions was intended just to locate the source of the problem, not to force you to use them permanently.

Anyway, I think that I managed to reproduce the issue and I will see if I could implement a workaround for it.
No worries, I wanted to be sure I was clear in my responses to you. Happy to try anything you suggest. Thank you for taking the time to understand!
By the way, the Magic Nodes' Path field is almost the same as the database field from the Songs table, with the addition of the drive letter taken from the Medias table. However, the MN's Folder field doesn't have a corresponding database field, but it is parsed from the Path field. So, when you apply any of the parse qualifiers as Right Of on it, you are basically double parsing the same data, which is why it is slower than Path.
This is interesting, it fully explains why this phenomenon was occurring in my nodes. I switched some uses of Folder to Path in my other nodes, and they do refresh much faster now.

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by ZvezdanD » Thu Jan 14, 2021 4:43 am

benfucius wrote: Wed Jan 13, 2021 12:49 pm I have just tried this as a Magic Node instead of a Playlist, and it does work without MM crashing. I expected this. However, my goal is to create this as a Playlist,
Thanks for the detailed report. My suggestions was intended just to locate the source of the problem, not to force you to use them permanently.

Anyway, I think that I managed to reproduce the issue and I will see if I could implement a workaround for it.

By the way, the Magic Nodes' Path field is almost the same as the database field from the Songs table, with the addition of the drive letter taken from the Medias table. However, the MN's Folder field doesn't have a corresponding database field, but it is parsed from the Path field. So, when you apply any of the parse qualifiers as Right Of on it, you are basically double parsing the same data, which is why it is slower than Path.

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Wed Jan 13, 2021 12:49 pm

Did you try the same mask outside of the Playlists branch, e.g. inside the Magic Nodes folder?
I have just tried this as a Magic Node instead of a Playlist, and it does work without MM crashing. I expected this. However, my goal is to create this as a Playlist, due to the limitation of MediaMonkey that the Android app will not access (and therefore) sync any music files from the Magic Nodes branch. I must create them as Playlists to access and sync to my phone. Such is the pain necessitating Playlist nodes. Unless you know of a better way?
Why are you using the Filter string on the local level of mask (for the HAVING part of the query), when it is intended for the WHERE part of the query? Did you try the same Filter to apply on the global level of mask?
Yes, I've tried both, same result. I always do my testing in Magic Nodes node first, since it is faster, and then when I am sure the filters work, I change it to a Playlist child node.
In the first post you said that you have two local levels, one for artist and another for album, but the mask contains only the first local level. Do you have the same issue with this mask?
Apologies, while I was trying to fix it, I mistakenly posted a redacted version of the mask yesterday. I'll post the correct mask at the bottom of this post, including edits following your suggestions.
In the first post you said that you are using Right Until, but the mask has Exclusive Right Until. Do you have the same issue with both of them? What happens if you use Exclusive Right Of instead of Right Of?
Yes I just tried both. The result with non-exclusive is that I get all artists (both @~ and ~). My goal is to show only @~ in this particular playlist, because these are the ones I have full discographies of. Using exclusive produces exactly what I want to see, before it ends up crashing MM.
What happens if you replace '%@~%' in the Filter with '%@@~%' (since the "@" is an escape character for the SQLite LIKE operator)?
I've just tried this too, it seems to break it, I get an error "Magic Node produced an error". However, my original issue is not that I get incorrect information; my mask works correctly. It causes MM to fill the RAM.
Why do you use filtering by "@~" in the Filter when it is already applied by the Exclusive Right Until? The Exclusive Right Until (and all other parsing qualifiers) has the effect to the all level of the mask, not only on the level where it is specified.
I have now removed the second Exclusive Right, so that only the first/root one exists. I've tested the original issue, and it still occurs the same.
What happens if you use Path instead of Folder?
I tried switching all instances of "Folder" to "Path" in the mask, and interestingly Path processes nodes faster than Folder, however this also means that the RAM fills up much faster (~3 minutes instead of the 15 or so with Folder). So I switched all Path to Folder to see what the difference might be. It runs slower overall, but produces the same end result.

A quick video I made of the RAM filling, sped up 3x:
https://imgur.com/FXKjgea

A still image of the crash:
Image

The issue, from where I stand, seems to be that when Magic Nodes is populating Playlist child nodes, MediaMonkey will store all this data into RAM. MediaMonkey does not empty or cleanse it's RAM at any point, it keeps it all in RAM. After some time of populating a Playlist node that filters through ~8,000 subfolders, the RAM fills to the 3.5gb limit of a 32-bit application, and crashes.

Question: Would it be possible to tell/force MM to clean it's RAM storage between node populations, or at all?

My current mask, including your above suggestions. Produces the same end result as my original post.

Code: Select all

:[Discogs]:|Icon:31|Show tracks:No|Filter:<Path> Like 'E:\Music\Library =\%' AND <Path> Like '%@~%'|Child of:Playlists|Position:Child\<Path|Unknown:No|Right of:+\|Exclusive right until:@~\>\<Path|Unknown:No|Right until:\|Right of:@~\>

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by ZvezdanD » Wed Jan 13, 2021 2:32 am

benfucius wrote: Tue Jan 12, 2021 5:37 pm Mask:
Did you try the same mask outside of the Playlists branch, e.g. inside the Magic Nodes folder?

Why are you using the Filter string on the local level of mask (for the HAVING part of the query), when it is intended for the WHERE part of the query? Did you try the same Filter to apply on the global level of mask?

In the first post you said that you have two local levels, one for artist and another for album, but the mask contains only the first local level. Do you have the same issue with this mask?

In the first post you said that you are using Right Until, but the mask has Exclusive Right Until. Do you have the same issue with both of them? What happens if you use Exclusive Right Of instead of Right Of?

What happens if you replace '%@~%' in the Filter with '%@@~%' (since the "@" is an escape character for the SQLite LIKE operator)?

Why do you use filtering by "@~" in the Filter when it is already applied by the Exclusive Right Until? The Exclusive Right Until (and all other parsing qualifiers) has the effect to the all level of the mask, not only on the level where it is specified.

What happens if you use Path instead of Folder?

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Tue Jan 12, 2021 5:37 pm

ZvezdanD wrote: Tue Jan 12, 2021 3:10 pm Please post the full mask that causes that issue. What is the version of Magic Nodes including its build number that you are using?
Mask:
:[Discogs]:|Icon:31|Show tracks:No|Child of:Playlists|Position:Child\<Folder|Unknown:No|Right of:+\|Filter:<Path> Like 'E:\Music\Library =\%' AND <Path> Like '%@~%'|Exclusive right until:@~\>

Version: v5.2.1.003

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by ZvezdanD » Tue Jan 12, 2021 3:10 pm

benfucius wrote: Tue Jan 12, 2021 12:00 pm I've done what I think is necessary to optimize it, and I understand that the memory issue belongs to MM and not your addon, however I'm open to suggestions on how to optimize or workaround this issue.
Please post the full mask that causes that issue. What is the version of Magic Nodes including its build number that you are using?

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by benfucius » Tue Jan 12, 2021 12:00 pm

I have a question for the dev of this plugin. First of all, fantastic plugin! I use it daily for multiple purposes.

I'm running into an issue where a Magic Node that I want to create is causing MediaMonkey to absorb the full 3.5gb of RAM that a 32-bit application is able to use. MM then crashes, leaving the Magic Node unfinished.

I've done what I think is necessary to optimize it, and I understand that the memory issue belongs to MM and not your addon, however I'm open to suggestions on how to optimize or workaround this issue.

In case it helps to explain the issue, I have my library generally organized like this. The symbols at the end I have manually put into the folder names, to help with node differentiation:

1. Library =
2. Genre +
3. Artist ~ (non discography), or Artist @~ (full discography)
4. Year cluster # (sometimes it makes sense to cluster albums or releases by year, this denotes that. Not always present)
5. Album(s)
6. Songs

Example: E:\Music\Library =\ELEC- Trance +\[A State Of Trance] ~\2020 #\947\(songs)
Example: E:\Music\Library =\LYRC- Rock Progressive +\Dream Theater @~\1994 - Awake\(songs)

Magic Node structure:

Image

- Level one is just a manual label.
- Level two displays Artists names, but only if it is a full discography. I differentiate these artists by putting a @~ at the end of the folder name.
.........Filters: <Path> Like 'Library =\%' AND <Path> Like '%@~%' (the idea here is to limit the entire node to the library folder, and only to folders with the discography symbol)
.........No boxes checked on the right, such as "Show files in filelist".
.........It also utilizes the "Right of" and "Right until" so that the nodes populate as album names.
..................Right of: +\ (end of the root folder)
..................Right until: @~\ (shows the name of the folders underneath these folders only)
- Level three displays Albums names.
.........No filters used here. Everything is filtered by level 2.
.........Only boxes checked on the right are "Show files in filelist" and "Show nodes"
.........I utilize "Right of" and "Right until" here too, so that the nodes are filtered to just the album name itself.
..................Right of: @~\
..................Right until: \

This is what it looks like when the node starts building (before the crash):
Image

After a while (approx 15 minutes), especially because some artists have many albums, MediaMonkey will crash like this:
Image

Re: help with custom node - artists only in compilations

by ZvezdanD » Wed Jan 06, 2021 1:37 am

timepiece wrote: Tue Jan 05, 2021 6:30 pm artist does not have any tracks where artist is also album artist.
I am not quite sure that understand your requirement, but if you need to find just tracks that have Artist different than Album Artist, then there is a predefined mask "Artist does not equal Album Artist".

If you want to find Artists that don't have any such track at all, maybe you could try this:

Code: Select all

(SELECT Count(*) FROM Songs AS T1 WHERE Songs.Artist = T1.Artist AND Songs.Artist = T1.AlbumArtist) = 0

help with custom node - artists only in compilations

by timepiece » Tue Jan 05, 2021 6:30 pm

I've been trying to get this for a while, and have been only partially successful.

I want to get all the artists I have who *only* have tracks in compilations. So, album artist is Various Artists or Movie Soundtrack etc, artist has at least two tracks, artist does not have any tracks where artist is also album artist.

I can get the first two criteria, but is the last one even possible?

I have a lot of compilations under at least 6 album artists (I separate by movie, tv, game, and broadway soundtrack, plus a couple more).

Re: Magic Node Help

by ZvezdanD » Fri Dec 11, 2020 3:33 pm

kbeck wrote: Fri Dec 11, 2020 3:15 pmBest I could tell the field I was looking for is Original Date, Although there is an Original Year. Tried both and got an error after closing.
I said "OrigYear":
Songs.SongTitle IN (SELECT SongTitle FROM Songs WHERE Length(SongTitle) > 0 GROUP BY SongTitle HAVING Count(DISTINCT OrigYear) > 1)

Re: Magic Node Help

by kbeck » Fri Dec 11, 2020 3:15 pm

ZvezdanD wrote: Thu Nov 19, 2020 6:02 pm
kbeck wrote: Wed Nov 11, 2020 10:30 am Need a list of tracks with the same title but have different Original Dates.
"Tracks with same Title from different Artists", replace Artist with OrigYear.
Thanks. I finally got a chance to try your suggestion. Best I could tell the field I was looking for is Original Date, Although there is an Original Year. Tried both and got an error after closing.

Re: Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

by ZvezdanD » Tue Dec 01, 2020 6:37 pm

Haaden2 wrote: Tue Dec 01, 2020 4:24 pm I get the error, "Error executing script event. Expected ')' in regular expression.
Thank you for the report. It will be fixed in the next version.

Top