Only tiny percentage of songs read into collection

Report bugs & feature requests for MediaMonkey Server and learn about the newest builds.

Moderators: drakinite, Gurus

allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Only tiny percentage of songs read into collection

Post by allwi »

Hi,

I tested MMS 0.4.3 and aside from one error which is reported on the console, everything seems to work.

But when I look more closely only a tiny percentage of the songs are actually imported by MMS.
The log shows

Code: Select all

Scan music directory: file:M:/Library, 26177 files to process
Caught exception: TypeError: SQLite3 can only bind numbers, strings, bigints, buffers, and null
The number of files is about right - the folder tree contains 26,723 files but this includes also cover pictures.

When I look into the MMS DB, it contains only 1,026 entries.

My music library is organised as follows

Code: Select all

Library
+ 0-9
  + [Album Artis]
    + [Album]
      + Music tracks
+ ABC
  + ...
+ DEF
...
+ YZ
...
The MMS database contains
  • All songs of the 5 artists in the 0-9 directory tree
  • Some songs of the alphabetically first 14 artists in the ABC directory tree
  • Some songs of the alphabetically first 12 artists in the DEF directory tree
  • No songs from any of the other directories
After I created the collection, and triggered a rescan, I've waited for 4h before I checked. The MMS server is sitting idle, i.e. consumes very little CPU.
Any ideas?
Ludek
Posts: 4875
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

I guess that it must have somehow stopped on a problematic file?
Veryfing the code and let you now...

Could you attach content of the "Verbose messages" and "Debug messages" of the logs here:
https://www.dropbox.com/s/kw9cvg56937yq ... 4.png?dl=0

EDIT: I replicated the "SQLite3 cannot bind error" on one testing file and looking into it:
https://www.dropbox.com/s/8lu4dgiiwvwzx ... 7.png?dl=0

EDIT2: Fixed as https://github.com/mediamonkeyserver/mms/issues/62
allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Re: Only tiny percentage of songs read into collection

Post by allwi »

Sorry for the late response. Thanks for looking into and (hopefully) fixing it.

If I don't want to build MMS from scratch, I'll have to wait for the next release, right?
drakinite
Posts: 963
Joined: Tue May 12, 2020 10:06 am
Contact:

Re: Only tiny percentage of songs read into collection

Post by drakinite »

Not necessarily. You can *run* MMS from the source code without building it into a release executable. There are instructions on the GitHub page if you want to try it.
Image
Student electrical-computer engineer, web programmer, part-time MediaMonkey developer, full-time MediaMonkey enthusiast
I uploaded many addons to MM's addon page, but not all of those were created by me. "By drakinite, Submitted by drakinite" means I made it on my own time. "By Ventis Media, Inc., Submitted by drakinite" means it may have been made by me or another MediaMonkey developer, so instead of crediting/thanking me, please thank the team. You can still ask me for support on any of our addons.
allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Re: Only tiny percentage of songs read into collection

Post by allwi »

Cool thanks, I'll give it a try
allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Re: Only tiny percentage of songs read into collection

Post by allwi »

Hmmm, I followed the instructions on Github, i.e. cloned https://github.com/mediamonkeyserver/mms.git, installed (i.e. build) and started MMS.
I can access the web UI without issues. However when I created a new collection for a mapped folder, MMS starts scanning but again stops after 2,613 files though it logs that it found 26,204 files in the directory tree:

Code: Select all

18:18   2613 files have been read from database in 1074 ms
...
13:04   Scan music directory: file:/var/churros/music/Library, 26204 files to process
I cannot see which version of MMS I'm running. Did I clone an older version?
Ludek
Posts: 4875
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Can you attach full logs from the web interface, i.e. both "Verbose messages" and "Debug messages" ?
allwi
Posts: 67
Joined: Sun Jan 22, 2012 7:34 pm
Location: Sydney NSW

Re: Only tiny percentage of songs read into collection

Post by allwi »

Hi Ludek,

how can I get the logs from the server? Only by showing them in the web UI, marking them and copying them to a text editor?
In this case, sorry I can't attach the full log because my browser crashed/became unresponsive.
But my understanding is that all log messages are stored in the DB, is that right?
In this case, please find the db here: https://drive.google.com/file/d/1VmwNy4 ... sp=sharing

I'm running MMS on Raspbian 9 (Stretch).
The music is on a NAS which is mounted via

Code: Select all

//192.168.1.100/Music on /var/churros/music type cifs (rw,relatime,vers=default,sec=ntlmssp,cache=strict,username=xxx,uid=0,noforceuid,gid=0,noforcegid,addr=192.168.1.100,file_mode=0755,dir_mode=0755,soft,nounix,serverino,mapposix,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
I confirmed that I can access the files, even the ones with the weirder filenames:

Code: Select all

pi@Clafoutis:~/MediaMonkeyServer $ ls -l "/var/churros/music/Library/YZ/Zoviet France/The Decriminalisation Of Country Music/"
total 88768
-rwxr-xr-x 1 root root     5311 Jan 11  2008 folder.jpg
-rwxr-xr-x 1 root root 17049970 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 01 Something Spooked The Horses.mp3
-rwxr-xr-x 1 root root 19196608 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 02 Electron Gate.mp3
-rwxr-xr-x 1 root root  2731524 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 03 Stainless.mp3
-rwxr-xr-x 1 root root 22569539 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 04 Pyroclastic Flow.mp3
-rwxr-xr-x 1 root root  2027681 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 05 Dust And Scratches.mp3
-rwxr-xr-x 1 root root   896684 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 06 Duct Tape.mp3
-rwxr-xr-x 1 root root  4429275 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 07 Purline.mp3
-rwxr-xr-x 1 root root 11794551 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 08 Spiitek.mp3
-rwxr-xr-x 1 root root  9953859 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 09 Light Abrasion.mp3
-rwxr-xr-x 1 root root   218756 Feb 12  2019 Zoviet France (The Decriminalisation Of ...) - 10 Light Abrasion.mp3
pi@Clafoutis:~/MediaMonkeyServer $ ls -l "/var/churros/music/Library/YZ/Zoviet France/The Decriminalisation Of Country Music/Zoviet France (The Decriminalisation Of ...) - 05 Dust And Scratches.mp3"
-rwxr-xr-x 1 root root 2027681 Feb 12  2019 /var/churros/music/Library/YZ/Zoviet France/The Decriminalisation Of Country Music/Zoviet France (The Decriminalisation Of ...) - 05 Dust And Scratches.mp3
pi@Clafoutis:~/MediaMonkeyServer $
Let me know, if you need something else.

Btw - when I looked into the logs, I saw the messages

Code: Select all

13:03:01   ==================================================
13:03:01   Connect using a web browser or using MediaMonkey 5.
13:03:01   MMS v0.4.3 running at http://192.168.1.104:10222 (or http://localhost:10222)
13:03:01   ==================================================
So it might be that I haven't downloaded the fix with the command

Code: Select all

git clone https://github.com/mediamonkeyserver/mms.git
sparhawk
Posts: 11
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

I just ran into this error today, running a new install of MMS pulled from git, run with NPM.
Feb 12 13:43:32 mms mm-server[13754]: Caught exception: TypeError: SQLite3 can only bind numbers, strings, bigints, buffers, and null
Here are my verbose and debug logs:
https://www.dropbox.com/s/rxohhul0ycwo1 ... g.txt?dl=0
https://www.dropbox.com/s/zsz7p813v7cit ... g.txt?dl=0
Ludek
Posts: 4875
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Are you sure you pulled the newest sources?

I've already fixed it as https://github.com/mediamonkeyserver/mms/issues/62
by this commit: https://github.com/mediamonkeyserver/mm ... 48d0178554

i.e. not part of the v0.4.3 yet, I guess I should ask Jiri to compile v0.4.4 with the fix.

EDIT: In your log I see that the last two files being scanned were:
/mnt/music/Alt, Folk, Pop 2/Mylène Farmer/[2009] C'Est Dans L'Air (Remixes 2)/Mylène Farmer - C'est Dans l'Air (Remixes 2) - 03 C'est Dans l'Air (Wize Remix).mp3
/mnt/music/Alt, Folk, Pop 2/Mylène Farmer/[2009] C'Est Dans L'Air (Remixes 2)/Mylène Farmer - C'est Dans l'Air (Remixes 2) - 01 C'est Dans l'Air (Extended Club Mix).mp3
--> so maybe if you could share these two files for analyzing to ensure it is (or is not) fixed?
sparhawk
Posts: 11
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

On February 12th, I got the source via

Code: Select all

wget https://github.com/mediamonkeyserver/mms/archive/master.zip
So, if that had the newest sources at the time of the wget, then that is what I compiled with npm.

Here are links to those two MP3 files:
https://www.dropbox.com/s/c9lgza7956ftf ... 9.mp3?dl=0
https://www.dropbox.com/s/6scluayyfguoj ... 9.mp3?dl=0

Thanks.
Ludek
Posts: 4875
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

Thanks, so if you put these files to a folder and scan the folder then does they fail to insert?

I've just tested on current sources and they were imported correctly:
https://www.dropbox.com/s/q4ofoli2p8r5a ... 1.png?dl=0
sparhawk
Posts: 11
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

Well, since I last posted, I've rebooted the server, and rescanned the directory, and those files show up okay now without an error, so :shrug:

However, a big chuck of files is still missing. My Media Monkey desktop software shows that the Alt, Folk, Pop collection has 127,692 files, but MMS only lists 32082 files across all my collections. Maybe it is still scanning?
sparhawk
Posts: 11
Joined: Tue Sep 06, 2011 3:35 pm

Re: Only tiny percentage of songs read into collection

Post by sparhawk »

Today, the database had 112,351 files in it, but it still stopped scanning before it reached the end of the collection, and never processed the next music collection I had. I can keep manually starting the scanning, but it would be nice to figure out why it stops before finishing.
Ludek
Posts: 4875
Joined: Fri Mar 09, 2007 9:00 am

Re: Only tiny percentage of songs read into collection

Post by Ludek »

So now your MMS collection has 112,351 files compared to 127,692 files in MM5?
Is the collection really the same? Don't forget that MMS collection can be defined only as a folder set (while MM5 collections have further criteria). Probably the log should show which files fails to scan and then sharing the files should help to resolve the issue.
Post Reply