Magic Nodes 4.2 w/ 380 masks & real GUI (2011-07-01)[MM2+]

[ZvezdanD]

Well now, don't make me blush I am just trying to be helpful.

By the way, there is one more, the most reliable method to get the exact number of the characters in the Comment, but it requires to enter the tags for the wanted track, e.g. Title (if it is not unique you need to enter some more tag e.g. Artist or Album or even better Path). So, install Bex's SQLViewer and enter:

Code: Select all

SELECT Length(Comment) AS Length, Comment FROM Songs WHERE SongTitle = '...'

[MM3 monkey]

Well now, don't make me blush I am just trying to be helpful.

By the way, there is one more, the most reliable method to get the exact number of the characters in the Comment, but it requires to enter the tags for the wanted track, e.g. Title (if it is not unique you need to enter some more tag e.g. Artist or Album or even better Path). So, install Bex's SQLViewer and enter:

Code: Select all

SELECT Length(Comment) AS Length, Comment FROM Songs WHERE SongTitle = '...'

That last post from you has gone so far over my head that I'm not even going to try and re-read it (again). I think the youths of this world might say: "WTF?".

Anyway, subtract and multiply or whatever it was... now we're talking. Not 199 characters, 203. There they are, all the comments that have been messed up [EDIT: but what about the ones that go onto more than or less than 3 lines; this could be endless].

Thanks, ZD. I can't comment further for a few hours but I'll report when I can.

[willyvds]

Zvezdan,
I have done some tests with MM and SQLiteSpy to detect where the slowness of MagicNodes comes from as you suggested. This was very usefull.
I get approximately the same response times with MM and SQLiteSpy so indeed MM or MN are not the problem. Nor is my PC...
It seems that the problem is caused by my husband....

What I mean: it seems that as soon as he is playing music with MM, the response times for MN on my PC drops terribly; for a specific mask: from less than 1 second when I am the only user to almost 10 minutes when he is also using the MM database on our NAS.

Does this make sense to you? Is there a way to improve this?

Regards, Willy

[ZvezdanD]

for a specific mask: from less than 1 second when I am the only user to almost 10 minutes when he is also using the MM database on our NAS.

Does this make sense to you? Is there a way to improve this?

Do you have the problem only with some specific mask? I see you mentioning something like that, so I need to ask that again. If this is the case, could you post that mask? Maybe MN didn't generate optimal SQL statement for it. However, if that mask is the same one which you posted before, then that SQL cannot be optimized since it is already simple enough. By the way, instead of (<Genre> <> "Klassiek") AND (<Genre> <> "Audiobook") you could write <Genre> NOT IN ("Klassiek", "Audiobook") - it is somewhat shorter and also could be faster.

[Lowlander]

when he is also using the MM database on our NAS

Are you saying that two PC's are simultaneously accessing a shared database on a third machine (NAS)?

[willyvds]

Zvedan,
the problem occurs generally, it's just that with this mask that I did the testing. I can replicate the timing difference consistently. But I do not really understand why a query takes so much longer if/when somebody else is just playing music.
Thanks btw for the suggestion.

Lowlander,
indeed we have 2 PC's and 1 network drive on which the MM database is; we do have two separate MM installations on the 2 PC's

Regards, Willy

[Lowlander]

In that case it might be limitations of SQLite as it isn't really designed for multi-user use.

[willyvds]

Some further testing showd that MN becomes slow if another instance of MM is running on another PC. It doesn't even have to play music.
I find that a bit odd.
Why would MN become slow simply because of MM running on another PC? Does MM keep the database locked even if not playing music?
Could I do something to prevent that?
Regards, Willy

[ZvezdanD]

Why would MN become slow simply because of MM running on another PC?

First of all, you should find out if only MN is slow or MM is generally slow. I told you already: firstly, you should open the Tools > Options dialog box and on the Magic Nodes sheet you should check "Allow editing of SQL queries:", "when expand field nodes" and "when select a field node". Then you should expand or select some Magic node and when you get the dialog box with the SQL statement, you should copy it (the SQL expression which you get when you select Magic node is just WHERE part of the SQL statement, so you need to add SELECT * FROM Songs in the front of the copied text). After that install Bex's SQLViewer add-on, paste the copied SQL statement into its text box and click on the Run Select Query button.

1. If the copied SQL statement is executing much faster in SQLViewer, then there is a problem with creating nodes by Magic Nodes, not with SQL;
2. If it is executing for similar time, then there are two another possibilities:
2a) SQL statement is not optimized very well by MN (well, by myself);
2b) the problem is because of the program itself.

You could also turn on the option "Show elapsed time in the progress bar after SQL queries" on the Magic Nodes sheet in the Options dialog box, so you could see the actual time of the execution. There are displayed 3 time values in the progress bar after expanding some Magic node: the first value is for SQL of the Unknown sub-node (if that node is enabled), the second one is for SQL of all other sub-nodes and the third one is for creating nodes and their adding to the tree. Which time value is longest for you?

Do you have problem when expanding Magic nodes or when selecting them or in both cases? Do you have similar problem with auto-playlists? How about Tagging Inconsistencies or similar add-on which creates nodes based on SQL?

[willyvds]

You ask a lot of question... well, I'll try to answer them:
- I did all the things you suggested, in MM and I also installed SQLiteSpy
- MM/MN and SQLiteSpy are equally fast or equally slow; it all depends on if I am the only MM user on our network
- It is not a matter of optimization: if I am the only user, then even our oldest PC is very fast (less than 2 sec for fully expanding a mask with 2 levels of subnodes), but as soon as MM is running on another PC - performance drops dead (5 minutes instead of 2 seconds); this happens also when the other PC is jus running MM and is not even playing a song
- The slowness occurs specifically if I click on the mask itself and when I click on the + before it; once the sub-nodes are visible, going further down goes a lot faster
- After the experiments I did, I think it is a general slowness and not just a MN slowness. It's just that with MN one is waiting for the result and then you are much more aware of the slowness
- Lowlander says it might be limitations of SQLite; well that could be it... but I don't understand why this already causes problems if the second PC only runs MM without playing music or anything; does MM keep the database locked all the time? But then why does MN give results in the end? I mean if de DB were locked by PC 1 - then PC 2 would not give any results, would it?

Does this answer your questions?

Regards and thanks for looking into this,
Willy

[ZvezdanD]

Does this answer your questions?

Well, not exactly. Firstly, you need to turn on the option "Show elapsed time in the progress bar after SQL queries", then tell me which of the mentioned 3 times is longest in your case.

Another questions that you didn't answer: Do you have similar problem with auto-playlists? How about Tagging Inconsistencies or similar add-on which creates nodes based on SQL?

If you have the same problem with auto-playlists or another add-ons, then why are you reporting this in the Magic Nodes thread as its fault? You should report it in the Bugs forum.

Yes, SQLite is not specifically designed for multi-user usage, but it should not cause such slowness as you are reporting. I am afraid that your NAS is the source of problems or maybe the network. Do you have similar problem with any application when accessing the same file with two computes?

Maybe you could try this:
- download some Live Linux distribution for USB;
- download some SQLite program for Linux;
- boot computer from that USB, start that SQLite program, open the MM.DB file on NAS and see if it is slow.

If you have BartPE or similar Windows Live distribution for USB, you could try the same using SQLiteSpy. You see, maybe your currently installed Windows is the source of problems.

[willyvds]

Zvezdan,
please note that I did not report the slowness as a specific MN problem; my initial question was about the best hardware configuration for MM in a multi-user environment. I put this question in the general help forum. However my question was moved to this thread, because I mentioned MN in the question. I did mention MN simply because when using MN, the slowness shows clearly. But essentially, we experience slowness whenever we use MM simultaneously on 2 PC's.
I am glad that you agree with me that this slowness is not necessarily explained by the lack of multi-user functionality of SQlite. Your repsonse gives me some new ideas to explore. I will need a few days to sort this out.
Thanks for the suggestions, I'll be back here when I have new facts.
Regards, Willy

[ZvezdanD]

please note that I did not report the slowness as a specific MN problem

Well, maybe latter, but here is a part of your first post: "I use MagicNodes a lot ... However I find it very slow, sometimes even very very very slow." And the subject of the mentioned thread that you posted in general help forum is "Speed up MagicNodes".

Anyway, I have a few more questions and one more idea what you should do. Do you experience slowness on both computers with MM or only on yours? If it is only on yours, what happens when you start MM first on your computer, then your husband on the second computer? Is it matter which MM is started first? Another idea is to exit MM and start SQLiteSpy on both computers and try to access MM.DB file in the same time to see if it has the same slowness in that case. If SQLiteSpy started on both computers and accessing the same MM.DB works fine, then MM has the problem.

[willyvds]

Zvezdan,

By saying I wanted to speed up MN on our PC's I didn't mean to say that I thought MN itself was the problem. But obviously that is how it seemed.
Sorry if I caused confusion or sounded offensive. That is certainly not my purpose!
I'm a big fan of MM and MN, and I only want to ask for support with a problem we have and and by doinig so to help improve MM and MN.
I am really glad you're looking into this.

To answer your questions about the timing (in seconds).
1) Only my PC, only MM: 0.72+0.38+0.02
2) Other PC also runs MM: 70.53+41.00+0.00
3) My PC runs MM + SQLiteSpy: 0.73+0.36+0.00
4) Other PC runs SQLiteSpy, not MM: 60.41+40.92+0.02

My husbands PC shows similar times if I do the test from his PC.
The order of switching the PC's and programs on and off doesn't make much of a difference.
I have to restart MM on my PC to get the faster times again after I stop MM or SQLite on the second PC.
Indeeed, not only MN suffers from this phenomenon. Also auto-playlists show the same kind of problem.

To summarize (as I see it): performane does not drop if 2 applications from 1 PC access the DB at the same time, but performance drops if 2 PC's connect to the DB simultaneously.
This happens once another PC has connected to the database; the second PC does not have to actually query the database and stopping the 2nd PC does not bring back perfomance.

I think it is save to say that this is not a MN problem, and not even a MM problem.
The question remains if I could do anything to improve the performance (another network drive, another operating system, ...)?

Any suggestion is highly appreciated!

Regards,
Willy

PS I have now deselected the option "show [unknown] node"; that does not solve the fundamental issue, but makes this mask a lot faster. Thanks for the suggestion.

[ZvezdanD]

The question remains if I could do anything to improve the performance (another network drive, another operating system, ...)?

You should try my last suggestion: close MM on both computers and start SQLiteSpy on both computers, open the same MM.DB database from NAS, then double click on the Songs node in SQLiteSpy on both computers and see how much time it takes to show the records. After that close SQLiteSpy on both computers and start it again on one of them, then try again the same with opened database and Songs table. If it works equally fast in both cases, then MM could have the problem.

Maybe you should try to store MM.DB on one computer and access it with both computers. Do you have some friend with NAS to borrow it to you for test sake? If you have the third computer, you could try it instead of NAS, again just for testing purpose. If anything of that works fast, then your NAS is problematic.