GPM integration [#15002][#14983][#15023][#14980][#14981][#14668][#14955][#14958][#14959][#14936][#14933][#14934][#15072]

[Barry4679]

re latitude tolerance, from your screenshots I see that some tracks has duration difference more than 40 seconds? How it this possible?

I wish that I knew exactly how|why it has happened.

All I did was allow Google Music Manager to upload from my flac tracks. ... That is what they have delivered. ... It is fit for purpose. The upload was reasonably efficient.

The material that I posted up here before indicated that Google may substitute with their existing "duplicate" track versions to minimise the upload task. You did not observe this when you uploaded to GPM, but maybe it does happen. I have read that people with low resolution tracks get 320bps versions in their GPM library, so it must be happening in those cases. I believe that it probably also happened sometimes during my upload because I had a low upload cap to my Internet access plan at the time, and although the upload seemed like it took a very long time, I think that I must have experienced some saving. ... And|or maybe they do something with leading and trailing silences when they transcode from flac to mp3 ... presumably this transcoding happens before upload.

So I don't know the exact reason .. all I can say is that the result was fit for purpose. I was not aware of it until MM5 disclosed the difference.

I think that we need to compare also track length, e.g. I have track called 'Track 12' in my library (empty album + empty artist) and such a track was false matched with another track called 'Track 12'. The duration comparing is helping to resolve this. But maybe we should do the track length comparance only when the album+artist info is not available (like in this case). I've re-opened https://www.ventismedia.com/mantis/view.php?id=14934 to tweak this further.

That sounds a good plan.

BTW: What app you are using that you are able to see the Comment tag? I don't see it in the GPM Chrome's extension and I also don't see it in the Google Play Music Desktop player: https://www.dropbox.com/s/oat8l9jrzv4bb ... 8.png?dl=0
I thought that Comment is always hiddedn, therefore I used it to store ventis metadata JSON.

What I see looks similar to what your image shows. ... have you tried scrolling down with that scroll bar I see there on your image

I am using chrome, but maybe I don't have their extension.

I right click the Google Music Manager icon in the hidden tray on the Windows task bar, take option to "Go to music Player". It opens https://play.google.com/music/listen?hl=en#/home

The display is from the Edit Info option on a tracks right click menu

Re: quality of uploaded tracks, you can set up the quality in the Options > Auto-conversion
We should probably just set up a higher default quality? Entered as https://www.ventismedia.com/mantis/view.php?id=15002

My defaults values show as being different to the image that you have linked in that Mantis report ... but the MM5 upload to my GPM library was using something like your values, ie. 140 VBR?. .. ?

https://www.dropbox.com/s/t8p5pgjzqxrgv ... s.png?dl=0

I will react on the further items later...

... thanks ... waiting, with interest

[Ludek]

I right click the Google Music Manager icon in the hidden tray on the Windows task bar, take option to "Go to music Player". It opens https://play.google.com/music/listen?hl=en#/home

Hmm, using the same way the comment tag is still missing:
https://www.dropbox.com/s/0qfpsie8zf1uu ... 1.png?dl=0
but probably not a big deal either way.

My defaults values show as being different to the image that you have linked in that Mantis report ... but the MM5 upload to my GPM library was using something like your values, ie. 140 VBR?. .. ?

You need to go into Devices & Services > Google Play Music > Options > Auto-conversion
Your screenshot seems to be taken from Tools > Options > Media Sharing > ...

[Barry4679]

Hmm, using the same way the comment tag is still missing:

I think that this is the answer: https://play.google.com/music/listen?hl ... ntsettings

https://www.dropbox.com/s/ulb6cr6tcsecn ... s.png?dl=0

I probably set that on some time ago, and then forgot about it.

but probably not a big deal either way.

I agreed, even if that Lab setting is where on by default.

[Ludek]

Ahh, I see, it is the 'View Track Comments' switch, thanks.

Enable a comments field to be seen and edited for all songs. Although many people ignore this field, some find it helpful for storing additional information such as multiple genres, moods, musicians, conductors, dance styles, etc.'

[Barry4679]

2117: It is a lot better. My non-classical music is now close to clean.

But there is still an issue which is peppering my classical tracks with quite a lot of duplicates.
Here is an example. https://www.dropbox.com/s/zzgag67g2c885 ... s.png?dl=0
Tracks that were within your duration latitude are matched OK, but tracks with a greater duration difference are still duplicated.

It mostly affects mostly my classical music, because that is where I most often used multiple artist tags (Conductor, and Soloist, and sometimes Orchestra).
Google just seems to have the conductor.
Since there is one artist tag in common you match these, but only if they are within your latitude?

I thought that you said that you were going to dispense with duration checking where the track had non-blank Album & Artist tags? ... This doesn't seem to be applied for tracks with multiple artist tags.

It would make a significant difference if this got fixed. ... I have many such examples

It is hard to see what else is going on, because most duplicates that i detect, and chase down, are caused by the above situation.

I see a couple of instances which suggest that there may still be duplicates caused by special characters. Here are two; / and $. There could be more. I could check for these easier if you fixed the 1st problem.
https://www.dropbox.com/s/wkdtsz7dru6zq ... 1.png?dl=0
https://www.dropbox.com/s/qck946fd2fpw0 ... 2.png?dl=0

The remainder of the duplicates are caused by minor tag differences.
You could clean up a lot with some fuzzy logic if you thought it important enough.

eg. strip off leading & trailing blanks before comparison, or during a 2nd pass after a first pass had done all the heavy lifting. See my version of this album has trailing blanks. Fixing this may avoid bug reports and complaints later on, because the difference is near-invisible.
https://www.dropbox.com/s/7ripcd91rpvf ... k.png?dl=0

Other common differences are & vs and .. or () vs [] vs {} .. or missing punctuation or quotes ...... you could strip these elements off before comparison ... you could strip of any suffix contained in a brackets of any kind, eg. [bonus track] or (take 2) or {with Fred}

[Ludek]

Thanks for reporting, re-opened https://www.ventismedia.com/mantis/view ... 934#c50855 for the matching improvements

[Barry4679]

I received a error during GPM->MM5 sync. "Conversion of file xxx failed":
https://www.dropbox.com/s/6a2a7j0djmp0i ... 2.png?dl=0

Download scanned files is not checked.
Why is is it trying to convert anything?? ... Can you please explain what gets "converted".
The status bar showed that it "converting" about 9 tracks ... it happened after it processed all the tracks, and then did the 2nd scanning phase ... just before it processed the playlists ... it was changing so quick in the status bar that i couldn't read it properly (!!We need a sync log!!) .. only this track had an error apparently.

I can't see anything that strange about the track ... ie. why it would trigger a "conversion" ... or why it would have an error.

I looked at it in MM4. It is not part of any playlist other than "Accessible Tracks".

I don't recognise the path mentioned in the error dbox (ie. \Music\...)

I looked at the track in the MM5 db. It has three entries for GPM in the DeviceTracks table which seems strange.
https://www.dropbox.com/s/7ais0fnk8nxuo ... e.png?dl=0

[Barry4679]

If is difficult to check the status of MM5 synchronisation to Web track locations for any specific track.

Local files files are visible via EntireLibrary|Location|Network.
Files that are remote are visible via EntireLibrary|Location|SomethingElse ... the delay indicates that this list is built via an Internet interrogation, not a local database query.

If I saw some anomaly during playback, how can I check whether the remote file is know to MM5?
eg. I am playing from the remote version of my tracks, and a track was skipped, or could not be queued. ... I can see that the local track is indexed by MM. I can see that it is also in my remote repository. .... But I can't see whether you have managed to update the MM database with keys & credentials for that track. ... Can I?

If I want to disconnect a MM track entry from a web location, how do i do that?

I think it that the MM5 track properties dbox should display the location(s) where the track is stored. Maybe a new tab for track location(s) (ie. additional to Basic, Details Classification, etc). Or a list on one of the existing tabs. ... Something which would list the local and web locations where the track is stored. ... It would be good if we could also uncheck a web location, to disconnect the track from the web location.

I see most track attributes via the MM track properties facility. It doesn't make sense that I can't see at which remote locations the track is stored.

BTW EntireLibrary|Location|GooglePlayMusic response is at a crawl. I have been waiting 30 minutes and it has just reached 15,000 tracks counted. Google has a problem ATM or 2117 has a problem? I just checked my Internet response. It is good.

[Ludek]

The tracks needs to be converted to MP3 format as GPM accepts only MP3s.

Re https://www.dropbox.com/s/7ais0fnk8nxuo ... e.png?dl=0
That's really strange, especially the DevicePath for GPM is expected to be GUID (the Google ID), not a path. i/e/ the third row shouldn't be there at all. To be found how the path could got there.

EntireLibrary|Location|GooglePlayMusic is very fast here, it just takes the tracks from the database (by SQL query) so there is no need for slowdown.
If you restart MM5, does it still takes so long to load the content? If yes, could you generate debug log?

[Barry4679]

The tracks needs to be converted to MP3 format as GPM accepts only MP3s.

?Is that relevant? My scan was triggered by File|AddRescanFilesToLibrary .... I assume that the "library" being added to is my mm5 "library" ... why is it trying to update my GPM "library"?

I presume that Devices&Services|ScanToLibrary would be the same thing ... ie. only applying updates to MM5 ... ie. not updating my GPM library ... btw, is Rusty going to agree to make the direction of the flow, in the tab heading, more clear?, especially in a GPM context, where both source and target ate "libraries".

That's really strange, especially the DevicePath for GPM is expected to be GUID (the Google ID), not a path. i/e/ the third row shouldn't be there at all. To be found how the path could got there.

It is reproducible. I cleared MM.db from portable directory, and reran everything ... same thing happened.

EntireLibrary|Location|GooglePlayMusic is very fast here, it just takes the tracks from the database (by SQL query) so there is no need for slowdown.
If you restart MM5, does it still takes so long to load the content? If yes, could you generate debug log?

Rebooted PC ... is still slow ... I put the stop watch on it as it passed 4,000 rows ... the duration from 4,000 rows to 10,000 rows took 6 and a half minutes, which is why I assumed that this was an Internet query. It gets slower as it progresses. ... Now that I think back, prior to 2117, it did not used to be this slow.

I will PM you a link to the dbgview log file.

Since I have had to crash MM5, with the Windows Task Manager, a couple of times due to freezes, I thought that the MM5 db maybe corrupted. I did a File|ManageDatabase with optimise & rebuild ... it did not improve things.

Even if it was lightning fast, I still think that the Track Properties dbox (Ctrl+Alt+P) should list the track location(s), along with all the other track attributes. ... you don't agree?

[Ludek]

I entered/updated the issues for Rusty's revision:
https://www.ventismedia.com/mantis/view.php?id=15023
https://www.ventismedia.com/mantis/view ... 668#c50009

I am going to look into you debug log and the track pairing issue (unexpected DevicePath), I am just struggling with some further urgent issues so I don't know whether I will be able to look into it today (and I am not here next week), but I will get back to you then.

Thanks for your input.

EDIT: re your log via PM: I see, you are in 'Album + Tracks' view and that's slow, I can replicate and entered as https://www.ventismedia.com/mantis/view.php?id=15024
Workaround is to you 'List View' until it is fixed.

EDIT2: 15024 will be fixed in 2119

[Barry4679]

thank you for keeping me in the loop

https://www.ventismedia.com/mantis/view ... 668#c50009

Interesting. It is slightly comforting that it is not only me that gets confused about MM5 sync and scan functionality. You guys also seem to be taking a circular path, going around and around it, while trying to come to a common understanding.

I read it all. And I have been playing with it for a few weeks. But can't say that I fully understand the topic yet. ... I don't think that whole package is quite fit for use yet.

Nomenclature

I don't know what "scan" means to you, but to me it has several meanings: to look at something carefully, maybe taking some measurements ... or confusingly, it can also mean that you looked through something quickly, to get the general overview, but you didn't read it in detail.

So when you use "scan" in MM5, I have no idea whether you actually did anything after looking at it. ?

So what does "SCAN to library" mean
And how is it different to "SYNC to target"

And in a GPM context "scan to LIBRARY" doesn't tell me what may be updated, as both ends are a "library"
And "sync to TARGET" is not much better ... I did expect that you would be writing to a "target" , but which end is the "target"?

And even "SYNC" is not clear to me ... what is going to be synchronised? ... just the hidden Google tracks and album IDs ... and my track tags? ... which tags? including my track ratings? .. including play counts? overwrite or additive? .. will it replace an existing tag value with a null value? ... album art? ... how does it handle tracks which exist only on the source side, does it ignore these, or does it insert tracks into the target ... what about tracks missing from the source, could it delete them from the target? ... if something is "sync'd" into MM db, are the tags in the music tracks also updated?

These are all important considerations for someone who has laboured over the perfection of a large MM db and music collection (ie. your typical MM customer ) ... It may stop them trying the new feature. They may be unhappy if they do try the feature, because in the case of a sync from GPM -> MM they don't really know what they will be receiving.

And I still don't understand why you would be converting my flac tracks to mp3 during a sync or scan TO MM.

Use Cases

I suppose there are lots of Use Cases, but the ones that I can think of are:

1. Copy all tracks to a non-GPM web source; Aim: an offsite back up, or relocate all your tracks to the web, also somewhere from where you can sync a subset of tracks to a mobile device ... MM5 covers this well
2. Copy some or all your tracks to GPM; same Aims as above, as well as an additional location that you can stream from ... covered by MM5, but not well if you have already used Google Music Manager to upload your collection to GPM, see below
3. Extend your music "collection" by adding some of the 30+ million tracks which GPM streams, into MM5; Aims: browse these tracks inside your "collection" using MM, and|or play them with MM, and|or put them into playlists designed by MM5 ... covered by MM, but poorly IMO

#2 is not covered well IMO because maybe I just wanted you to link my existing GPM track versions, to my MM versions, so that i can play them with MM, and I can use them in GPM playlists that I will design with MM .... maybe I don't want you to add all the junk that is in my GPM library into my MM db ... maybe I don't want to soil my MM db with Google tags, and play counts, etc etc

Or maybe I did want to also sync some info from GPM, but I want to control over what is done ... eg. bring in just the google genres, and ADD in my GPM playcounts ... but don't touch tags such as Album, track & Artist names ... maybe because this will cause a disjoint in my Last.fm library next time I play the track, or maybe I just prefer my own tags

#2 is also not well covered as you don't have an exception report facility or workaround, where I can see which of my tracks are not linked to my GPM library ... ie. those that you could not match (presuming that I get a facility to stop you inserting duplicates) ... if I had an exception list I could adjust my MM or Google tags, so that you could achieve match up ... acknowledging that you have done a lot of work to minimise mismatches, which is really helpful, but you will never get to 100% ... eg. maybe i re-tagged something in MM after it had been uploaded to GPM

#3 is poorly covered IMO, because I only have two unpalatable options at the moment.

1. Add Everything that you can't match from GPM library. ... I like to add a bunch of albums to my GPM library, from the 30+ million tracks that Google have on offer ... play them once or twice, and then add just those that I want into MM. ... There is a lot of junk in my GPM library. There are tracks in my GPM library that I haven't played yet, or haven't decided upon. As well as tracks which granddaughter likes dancing to, or that I put there for a party. etc ... I want to control what gets added to my MM database.
2. Add selected tracks. ... This is my preferred option, but you don't support it well currently IMO. ... I would like to go Devices&Services>GPM>AllTracks, and then be able to filter that list, to show just tracks, or albums, which are not currently in MM ... at the moment you list all of my 40,000+ tracks, most of which are currently in my MM database

So hopefully you see at least some of these as worthwhile additions:

1. sync option from GPM>MM, which does no track additions
2. fine grained control over which tags get sync'd
3. exception list showing non-linked MM tracks, able to use right click menu to add selected tracks or albums to MM db
4. and maybe a MM->GPM option which populates the GPM comments field with your JSON string, so that GPM better serves as an offsite backup for anyone who used Google Music manager to upload

[Barry4679]

Matching with 2120 is now very very good.

Few duplicates. All those that I can identify are due to tagging differences with the Google track substitutions into my library as part of their upload minimisation, or library space rationalisation, polices.

The only other thing that I can think of, that you could do to assist, would be to have a final pass of unmatched entries, and try and obtain a match after excluding punctuation and spaces from both sides

ie. I see duplicates due to
artist: Gary Clark, Jr. vs Gary Clark Jr.
and track name: 'Senzo' - Contours And Time vs Senzo - Contours And Time
and Track Name: Oh, Pretty Woman vs Oh Pretty Woman
and Track name: Steppin' Out Queen <alternative Take> vs Steppin' Out Queen (alternative Take)
and Album name: Agharta, Disc 2 vs Agharta (disc 2)

It would help if the following could be stripped prior to tag comparison ,.:;'"<>[]{}() and spaces

User verification and tidy up after sync

The procedure that I would like to use and recommend would be:
1. sync GPM->MM, with option to add tracks turned off
2. run exception report, or filter a view, so that only non-sync'd tracks were listed (ie. tracks in MM db, which have no GPM ids)
3. manually re-tag either MM or GPM tracks to resolve the differences
4. rerun steps 1 & 2

I know that you haven't yet answered the requests in my last post. Without these tools it is virtually impossible to verify the sync, and identify & fix any issues.

• duplicated tracks are hidden amongst all the other added tracks, which were added because they exist only in the GPM library

• the non-sync'd tracks have no visible indication showing that they do not have their Google ID's stored

Duplicated or non-sync'd tracks may cause the following issues (I think?)

• MM library bloat

• tracks 'skipped' during playback from GPM

• tracks repeated during local playback since MM can see a album track which exists only at GPM (possible?)

• tracks which cannot exist to playlists to be sync'd to GPM

• support issues regarding concern that GPM sync may have "corrupted" the MM db

I would also like to redefine one of my requests

3. exception list showing non-linked MM tracks, able to use right click menu to add selected tracks or albums to MM db>>>

It would be good to have an exception list, or view filter, which identified MM tracks which have no GPM ID's

And it would also be good to have a Devices&Services>GPM>AllTracks list or filter, which identified GPM tracks which are not included in the MM db ... where the right click menu offers Add and Download options

[Corneloues]

Have just started using MM5 purely so I can try the GPM integration. I'm a long term MM4 Gold user and have been looking for another way to stream my own music since MSFT announced that Groove for iOS and Android was being discontinued.

To things:

1. I've added to GPM accounts: my own and my wife's. The only thing I can't do is name them at the moment so I don't know without looking at the sync settings which is which - Can we have the option to rename the profiles please.

2. Will MMA be able to stream from GPM. I would prefer to use the MMA app so would be good if the app could stream and download from GPM.

Cheers,

Roy

[Ludek]

1. I've added to GPM accounts: my own and my wife's. The only thing I can't do is name them at the moment so I don't know without looking at the sync settings which is which - Can we have the option to rename the profiles please.

You can rename it on this page by dlouble-clicking the title and pressing [Apply] button after renaming:
https://www.dropbox.com/s/re8gtm9312f93 ... 6.png?dl=0

But I agree that it is confusing as other items (e.g. playlists) can be renamed even in media tree (either by F2 pressing or right-click > Rename)
To be fixed as:
https://www.ventismedia.com/mantis/view.php?id=15072

2. Will MMA be able to stream from GPM. I would prefer to use the MMA app so would be good if the app could stream and download from GPM.

GPM isn't integrated to MMA yet, but it is planned. For now you have to use GPM app for Android.