problems during download from GPM [#15133] [#15693]

Post a reply

Smilies
:D :) :( :o :-? 8) :lol: :x :P :oops: :cry: :evil: :roll: :wink:

BBCode is ON
[img] is ON
[url] is ON
Smilies are ON

Topic review
   

Expand view Topic review: problems during download from GPM [#15133] [#15693]

Re: problems during download from GPM [#15133]

by Ludek » Fri May 24, 2019 6:45 am

Barry4679 wrote: Fri May 24, 2019 5:35 am I unplugged the Internet before altering one of these playlists, to test how MM5 reacted when unable to auto-sync to GPM.

The error message is not very user-friendly. https://www.dropbox.com/s/nbmd1r6ryvse4 ... e.png?dl=0

Also you could consider tuning? I changed just one playlist, but you auto-sync all of them. I have a playlist for each Genre; so there are 21 playlists.
I see, it wanted to create all the playlists, because it failed to list them (because of the interrupted internet connection) and thus MM5 thinks that they don't exist yet (in GPM), tracked as: https://www.ventismedia.com/mantis/view.php?id=15693

Re: problems during download from GPM [#15133]

by Barry4679 » Fri May 24, 2019 5:35 am

I unplugged the Internet before altering one of these playlists, to test how MM5 reacted when unable to auto-sync to GPM.

The error message is not very user-friendly. https://www.dropbox.com/s/nbmd1r6ryvse4 ... e.png?dl=0

Also you could consider tuning? I changed just one playlist, but you auto-sync all of them. I have a playlist for each Genre; so there are 21 playlists.

Re: problems during download from GPM [#15133]

by Ludek » Fri May 24, 2019 3:46 am

GPM changes are synced only when the user initiates a sync?
Yes, until you set up scheduled sync.


Also I logged MM out of my GPM account .... then altered one of these playlists ... then logged back in .... the playlist was not auto-synced.
Does the same thing happens if the user's internet connection is down or interrupted? ...etc etc
Yes, manual initiation of sync is needed (or scheduled sync) in such a cases.

Re: problems during download from GPM [#15133]

by Barry4679 » Thu May 23, 2019 7:50 am

Ludek wrote: Thu May 23, 2019 5:14 am Otherwise such a playlists are synced bi-directionally. i.e. changing them on GPM updates them in MM>GPM group and vice-versa.
What triggers the sync in the GPM->MM direction?
ie. if I use MM to alter one of these playlists, I now see that you auto-sync the change to the GPM copy. ... ie. there is no need for me to initiate a sync.

But how do you know to trigger a sync if I alter the GPM version of this shared playlist? ... I don't observe this happening ... are they auto-synced in a MM->GPM direction, but GPM changes are synced only when the user initiates a sync?

Also I logged MM out of my GPM account .... then altered one of these playlists ... then logged back in .... the playlist was not auto-synced.
Does the same thing happens if the user's internet connection is down or interrupted? ...etc etc

So maybe these need to thought of as (probably) synced playlists? ... Is this such a good idea? ... I am trying, but am slow to warm to it.

Or does this what already happens in a MM4|MMA context, and is likely to be expected behaviour by MM customers? .... I have only ever used MMA as a recipient, and have not tried syncing anything back.

Ludek wrote: Thu May 23, 2019 5:14 am As for the name of the group, I guess that "Google Play Music - auto-synced" isn't clearer.
What about "Synced from Google Play Music" ?
Or maybe "Scanned from Google Play Music" ?
I don't think that the names that you have suggested convey the special nature of these playlists.
Actually I would strengthen the name I suggested to "Google Play Music - changes made here are also auto-synced to GPM"

This dbox should be made more clear ... it should explicitly say that the change will also be made to the GPM version of this shared playlist.
https://www.dropbox.com/s/p1l56kmqf8vtk ... M.png?dl=0

Re: problems during download from GPM [#15133]

by Ludek » Thu May 23, 2019 5:14 am

Yes, the playlists after the "Google Play Music" group are the original GPM playlists scanned into MM library.
And if you don't touch them then they are never synced back to GPM again.
If you go under the "Google Play Music" group and change a GPM playlist content then YES, the change is synced back to GPM (supposing that the playlist is checked on the sync-list, which is by default).

As for the name of the group, I guess that "Google Play Music - auto-synced" isn't clearer.
What about "Synced from Google Play Music" ?
Or maybe "Scanned from Google Play Music" ?

Otherwise such a playlists are synced bi-directionally. i.e. changing them on GPM updates them in MM>GPM group and vice-versa.
They were added under this separate group in course of https://www.ventismedia.com/mantis/view.php?id=15396 -- reasons described in the Mantis issue

Re: problems during download from GPM [#15133]

by Barry4679 » Wed May 22, 2019 8:04 am

Ludek wrote: Wed May 22, 2019 4:14 am So it does not overwrite your MM5 playlists with GPM data.
The playlists under "Google Play Music" group are only the playlists scanned/imported from GPM. And you can uncheck them on the 'Library Content' page if you don't wish to sync them back to GPM.
I have just noticed that the playlist group (Google Play Music) that you created, seems to be auto-sync'd ... correct? ... ie. if use MM5 to add or remove a track (using the "remove from playlist only" option) from one of these playlists, It also updates the GPM version of the playlist ... ie. I don't need to manually run a GPM "Sync Now" to make this happen. ... correct?

I was assuming that these were static copies of playlists imported from GPM ... and that they were sync'd by the Sync Now button

After you confirm I will retest, as it probably explains what I was confusing me while I was trying to setup and run my test cases.

If this is how it works, it is not clear from the UI is it? ... Maybe the playlist group that you created should be named "Google Play Music - auto-synced" ... but I suppose that could be confusing as it is only a one way sync (ie. MM->GPM) .... ie. any change made in the GPM client becomes lost presumably?

Ludek wrote: Wed May 22, 2019 4:14 am
And I still don't understand why you produce the Scan Results info dbox prematurely. ... ie. the sync is still running ... it has synced tracks, but is still syncing playlists ...
It is "Scan results" box, i.e. it only includes scan results (as you previously requested number of times and was finally implemented as https://www.ventismedia.com/mantis/view.php?id=15360 ). MM5 scans the GPM at first (GPM >MM), then it sync the GPM location (MM > GPM) and shows also another "sync log" at the end (in case a failure happened during sync)
The Mantis ticket that you created says "Some users are requesting a scan log to be shown after the scan/sync operation.

I have always thought of the scan and sync as being two sub-steps within a single operation. ... from a user perspective the scan has no output does it?

OK, the dbox title is "Scan Results", but it also contains info about how many tracks updated and imported, ie from the sync stage.

It is confusing that that it doesn't also cover any activities done to playlists at the same time .... I manually started a single Sync operation, and the dbox seems to indicate the the process has completed ... but the wheel is still spinning, and I can see in the status bar that things are still happening.

Any way, I will drop it, as I can see you are getting irritated about it.

Re: problems during download from GPM [#15133]

by Ludek » Wed May 22, 2019 4:14 am

[UPDATE] What seems to be happening is that it is processing playlists in a playlist group in MM5 named "Google Play Music", and the direction of sync is GPM->MM....
Yes, this are playlists that were imported from GPM to MM.
If you checked '[x] Only include content that matches files already in the database' then according the tooltip:
'On sync, the database is only updated with links to files & playlists from GPM that match files already in the database'

.. which is exactly what MM5 did, it imported only the playlists that matches files already in the MM database.
i.e. it did not import playlist(s) including a GPM only track.

So it does not overwrite your MM5 playlists with GPM data.
The playlists under "Google Play Music" group are only the playlists scanned/imported from GPM. And you can uncheck them on the 'Library Content' page if you don't wish to sync them back to GPM.

And I still don't understand why you produce the Scan Results info dbox prematurely. ... ie. the sync is still running ... it has synced tracks, but is still syncing playlists ...
It is "Scan results" box, i.e. it only includes scan results (as you previously requested number of times and was finally implemented as https://www.ventismedia.com/mantis/view.php?id=15360 ). MM5 scans the GPM at first (GPM >MM), then it sync the GPM location (MM > GPM) and shows also another "sync log" at the end (in case a failure happened during sync)

Re: problems during download from GPM [#15133]

by Barry4679 » Tue May 21, 2019 9:00 pm

Ludek wrote: Tue May 21, 2019 5:37 am
Barry4679 wrote: Sat May 18, 2019 9:31 pm I have my GPM sync configured to just push GPM track|album links into my MM db, ie. no new track downloads, no sync GPM metadata to MM, nothing checked the D&S>GPM>AllTracks LibraryContent tab .... so why is the sync copying my MM playlists into my GPM library? .... why is it compulsory? ... it is annoying ... I have a lot of playlists ... I don't need them sync'd to my GPM library ...it takes a relatively long time ... it would be good if it was an opt in action IMO ... and it seems unacceptable that there is no Opt out config option for this step.
You need to have the MM playlists checked to be synced to GPM.
Just uncheck them in the 'Library Content' tab > Playlists
It is good that it should work that way, ... but it doesn't appear to.

ie. as I said, there is nothing checked on the Library Content tab ... I checked inside it's playlist node ... no individual playlists checked either.

[UPDATE] What seems to be happening is that it is processing playlists in a playlist group in MM5 named "Google Play Music", and the direction of sync is GPM->MM, which is the opposite to what I had assumed .... that group was not imported from my MM4 db ... you created that group, and imported the playlists from my GPM library? ... maybe you should not have done that, because I have "Download GPM content that is not accessible locally" unchecked.

MM5 is also updating a group of playlists that have been imported from my MM4 db, and which are also in my GPM library ... ie. syncing them from GPM->MM ... but when I hover over my config settings, the tooltip says "the (MM5) database is updated with links to files and playlists from GPM" ... it doesn't say that it will overwrite my MM5 playlists with GPM data

nb. maybe there should be two sets of config check boxes; one for playlists, and the other for the tracks ... some people may want to sync playlists ... but want to do this without polluting their local music collection with all the crap in their GPM library ... and other people may want to download some tracks, but won't be too happy if you overwrite their MM playlists with whatever they last sync'd to GPM

And I still don't understand why you produce the Scan Results info dbox prematurely. ... ie. the sync is still running ... it has synced tracks, but is still syncing playlists ... you jump in with the Scan Results dbox, which has stats for tracks which where processed, updated and imported ... but there is nothing covering playlists?

https://www.dropbox.com/s/nh7d2yc4wr5e4 ... x.png?dl=0

Ludek wrote: Tue May 21, 2019 5:37 am
Barry4679 wrote: Sat May 18, 2019 9:31 pm I got several of the "this message will self destruct in n seconds" dbox messages .... is this just a debug thing, please say that they would be permanent part of MM5? ... they are beyond annoying ... ie. something bad has happened ... you told that me, but I haven't enough time to read and absorb it ... there is nothing I can do about it anyway, and I don't have enough to copy anything down, or get a pic to send to you .... !!
The dialog is not self-destructing once you click on the (X) button on the line:
(x) Warning will desappear in 6 seconds

Otherwise the "self-destructing" dialog just inform about a problem during a long action, but at the end of the long action (like sync/download) there is a final log that is not self-destructing and lists _all_ the items previously shown in the self-destructing dboxes.
You could reduce user stress levels by making the message " Warning will disappear in 6 seconds, but will be included in the Error Summary at the end of the download task"

Also the messages don't seem to be actionable, nor comprehensible
https://www.dropbox.com/s/8a99kk77bzr5j ... s.png?dl=0

Re: problems during download from GPM [#15133]

by Ludek » Tue May 21, 2019 5:37 am

Barry4679 wrote: Sat May 18, 2019 9:31 pm MM5 version 2175: I retested downloads from GPM

I worked around the UI stability previously reported by switching away from Album&Tracks view mode. View mode, with Column Browser filtering to Source="link", and within that to AlbumArtist & Album, gives a useful view, and has a stable UI when downloading.

I loaded up the download queue (number of concurrent downloads remained at default of 4) ... I had some crashes

I missed the ID of the 1st crash, but the log was dispatched to you.

I started again and got crashes c6c6657c and 95477918 ... I had debug running for one of them, and I PM'd you a link to the log file.

In both cases the background queue was loaded up and was processing ... one of the crashes appeared to happen when I tried to display track properties for a track on Devices&Services>GPM>AllTracks while another album was downloading ... Once I saw the message "too many threads for indexing"

also these errors ... how we supposed to respond to error messages like this? ... it didn't seem to send a message to you ... there seems to be nothing I can action, and there seems to be no useful message to report to MM Tech Support?
https://www.dropbox.com/s/x5ul6rheulwrj ... s.png?dl=0

---------
Thanks, I will analyze the logs.

EDIT: Crashes c6c6657c and 95477918 are fixed in build 2177.
I also uncovered the possible hidden exceptions ( https://www.dropbox.com/s/x5ul6rheulwrj ... s.png?dl=0 ) in build 2177
So please re-test with 2177+

some other related observations:

I have my GPM sync configured to just push GPM track|album links into my MM db, ie. no new track downloads, no sync GPM metadata to MM, nothing checked the D&S>GPM>AllTracks LibraryContent tab .... so why is the sync copying my MM playlists into my GPM library? .... why is it compulsory? ... it is annoying ... I have a lot of playlists ... I don't need them sync'd to my GPM library ...it takes a relatively long time ... it would be good if it was an opt in action IMO ... and it seems unacceptable that there is no Opt out config option for this step.
You need to have the MM playlists checked to be synced to GPM.
Just uncheck them in the 'Library Content' tab > Playlists



I got several of the "this message will self destruct in n seconds" dbox messages .... is this just a debug thing, please say that they would be permanent part of MM5? ... they are beyond annoying ... ie. something bad has happened ... you told that me, but I haven't enough time to read and absorb it ... there is nothing I can do about it anyway, and I don't have enough to copy anything down, or get a pic to send to you .... !!
The dialog is not self-destructing once you click on the (X) button on the line:
(x) Warning will desappear in 6 seconds

Otherwise the "self-destructing" dialog just inform about a problem during a long action, but at the end of the long action (like sync/download) there is a final log that is not self-destructing and lists _all_ the items previously shown in the self-destructing dboxes.

Re: problems during download from GPM [#15133]

by Barry4679 » Sat May 18, 2019 9:31 pm

MM5 version 2175: I retested downloads from GPM

I worked around the UI stability previously reported by switching away from Album&Tracks view mode. View mode, with Column Browser filtering to Source="link", and within that to AlbumArtist & Album, gives a useful view, and has a stable UI when downloading.

I loaded up the download queue (number of concurrent downloads remained at default of 4) ... I had some crashes

I missed the ID of the 1st crash, but the log was dispatched to you.

I started again and got crashes c6c6657c and 95477918 ... I had debug running for one of them, and I PM'd you a link to the log file.

In both cases the background queue was loaded up and was processing ... one of the crashes appeared to happen when I tried to display track properties for a track on Devices&Services>GPM>AllTracks while another album was downloading ... Once I saw the message "too many threads for indexing"

also these errors ... how we supposed to respond to error messages like this? ... it didn't seem to send a message to you ... there seems to be nothing I can action, and there seems to be no useful message to report to MM Tech Support?
https://www.dropbox.com/s/x5ul6rheulwrj ... s.png?dl=0

---------

some other related observations:

I have my GPM sync configured to just push GPM track|album links into my MM db, ie. no new track downloads, no sync GPM metadata to MM, nothing checked the D&S>GPM>AllTracks LibraryContent tab .... so why is the sync copying my MM playlists into my GPM library? .... why is it compulsory? ... it is annoying ... I have a lot of playlists ... I don't need them sync'd to my GPM library ...it takes a relatively long time ... it would be good if it was an opt in action IMO ... and it seems unacceptable that there is no Opt out config option for this step.

Also, unlike the track download and track ID sync step, there is no info completion dbox nor log from the playlist sync step ... ?? how can see whether the sync completed or did anything?

I got several of the "this message will self destruct in n seconds" dbox messages .... is this just a debug thing, please say that they would be permanent part of MM5? ... they are beyond annoying ... ie. something bad has happened ... you told that me, but I haven't enough time to read and absorb it ... there is nothing I can do about it anyway, and I don't have enough to copy anything down, or get a pic to send to you .... !!

Re: problems during download from GPM [#15133]

by Barry4679 » Thu Apr 18, 2019 2:59 am

Ludek wrote: Thu Mar 28, 2019 8:45 am FYI: The "HTTP/1.1 403 Forbidden" errors and the self-paused items issue will be fixed in 2168.

As for the UI instability (as previously reported): Isn't the issue that with the tracklist filtered using ColumnBrowser with Source col = "link+cloud" the tracks that are just downloaded becomes "download+cloud" -- so they are correctly moved outside of the view (to the "download+cloud" source col).
If you want to select items one by one and initiate download in batches then I guess the better is to disabled Column Browser ( and potentially sort the entire tracklist by the 'Source' column like this: https://www.dropbox.com/s/rg2aywvanlmje ... 0.png?dl=0 )

The tracks are just re-sorted after the download, but the selection remains and the downloaded items are still in the view (as it isn't filtered by the download state - as in the case of the Column Browser example).

Is this what you are seeing?
It took me a while to get back to this.
As per your suggestion I turned the Column Browser off, and sorted the grid by the Source column.

This workaround doesn't improve grid stability. It is horrible during download. I can't scroll grid. Great difficulty doing anything all with the UI when a download batch is in progress. ... It would be better if the download was done via a modal dbox if this is as good as it gets.

I use view mode Album&Tracks ... have you tried that mode ? .... during the download batch the Artwork and Summary columns blink flash in and out of view as each track is downloaded ... this makes the the grid collapse and re-expand over and over again.

I got another occurrence of "HTTP/1.1 403 Forbidden" error .... ? maybe when I am trying to use the UI during the download batch, I pause the download, and something times out. ... the error message is not informative ... ie. doesn't show which track had the error. https://www.dropbox.com/s/8uzpxoxmf1gkf ... g.png?dl=0

it also generated a dump: 83902A60

I also got the annoying "this message will self destruct in n seconds" a few times again ... I caught a pic this time ..... so this thing jumps to the front of the z-order, and the grid is jumping all over the place .... and if I select a track and then shift click to select a block of tracks, 9/10 times I don't get the block that I want
https://www.dropbox.com/s/kin0oyal2sggw ... x.png?dl=0

Re: problems during download from GPM [#15133]

by Barry4679 » Sat Mar 30, 2019 5:40 am

Ludek wrote: Thu Mar 28, 2019 8:45 am Isn't the issue that with the tracklist filtered using ColumnBrowser with Source col = "link+cloud" the tracks that are just downloaded becomes "download+cloud" -- so they are correctly moved outside of the view (to the "download+cloud" source col).
[snip]
Is this what you are seeing?
Probably that is what I am seeing.

Maybe it is a minor problem, but I would think that it is a common Use case ... ie. filter to tracks available for download .... browse ... see something worth downloading, select for download ... more browsing, more selecting, etc.

the instability that I am talking about is as follows ... see a bunch of consecutive tracks that I want to download ... click top track .... navigate down and shift+click the bottom track ... but if your background task completes a track download between these two clicks, or while I am navigating down to prepare for the shift+click, the grid can jerk about .. and I either don't get a multi-track selection at all, or the tracks that I thought that I selected, are not the ones that are actually selected.

I can see what you are saying, but you wouldn't expect this (mis)behaviour from Windows Explorer would you?, even if some other process was adding or removing files to the directory that your were operating on.

Since this grid may be updated by some background task, maybe you shouldn't be recording either of my clicks by relative grid row position ... ie. use some other mechanism ... and don't jerk the grid around while I am trying to try scroll from row to row.

I think that this problem first started when I complained that the GPM AllTracks grid wasn't live at all ... maybe half-live would have been better? ... maybe the grid shouldn't be live while the download queue is being processed one track at a time ... just updated at batch end, at least that would minimise the issue

Or maybe, since the grid may be updated by a background process, F5 should be required to refresh the grid ... ie. is the cure is worse than the problem that it fixed?

I just checked MM4, and see that it has this "problem" also. .... but it is not so much of an issue there ... ie. if I do a File|AddScan .... I select all tracks or albums that I want to upload via check boxes in the sub node ... and then you do the upload to MM ... and I can see the task is still running, because MM4 is blessed with a proper status bar ... I intuitively know that MM is busy

In MM5 for the new GPM download, I don't have a sub node nor check boxes ... so I select album or track batches, and you start the download sub-task immediately ... and the background task is all but invisible ... ie. no progress in the status bar, and no spinning wheel either.

I will test your sort-column workaround some time soon.

Re: problems during download from GPM [#15133]

by Ludek » Thu Mar 28, 2019 8:45 am

FYI: The "HTTP/1.1 403 Forbidden" errors and the self-paused items issue will be fixed in 2168.


As for the UI instability (as previously reported):

Isn't the issue that with the tracklist filtered using ColumnBrowser with Source col = "link+cloud" the tracks that are just downloaded becomes "download+cloud" -- so they are correctly moved outside of the view (to the "download+cloud" source col).
If you want to select items one by one and initiate download in batches then I guess the better is to disabled Column Browser ( and potentially sort the entire tracklist by the 'Source' column like this: https://www.dropbox.com/s/rg2aywvanlmje ... 0.png?dl=0 )

The tracks are just re-sorted after the download, but the selection remains and the downloaded items are still in the view (as it isn't filtered by the download state - as in the case of the Column Browser example).

Is this what you are seeing?

Re: 2166: problems during download from GPM

by Ludek » Mon Mar 18, 2019 5:08 am

Re the UI being unstable as previously reported. ... I haven't been able to replicate this, but going to look into this (test this again) once I have a chance.

Re: the recoverable "HTTP/1.1 403 Forbidden" error, I haven't seen this, but going to check how this can happen and why the error box isn't shown immediately (as it should).

Re-opened as https://www.ventismedia.com/mantis/view.php?id=15133

problems during download from GPM [#15133] [#15693]

by Barry4679 » Sat Mar 16, 2019 6:51 am

If I were going to download tracks from GPM, I would want to firstly browse and select from amongst tracks which are in my GPM library, but have not been matched to my MM library .... ie. from Devices&Services>GPM>Alltracks, filtered using ColumnBrowser with Source col = "link"

That would be the recommended method wouldn't it? ... Or is there a better way?

The problem is that while a download batch is being processed, a grid which is filtered as described above, becomes unstable ... as previously reported. ... I have to fight with the UI if we want to select more tracks to add to the download batch.

The situation is made more difficult because the UI does not properly report that a prior download batch is in progress ... ie. no spinning wheel in the normal place on the tool bar ... and no spinning wheel at all, if the Media Tree is not displayed, or is scrolled towards the top, so that your temporary Media Tree Download node is invisible.

Another problem is that track download requests can become suspended, for no apparent reason, and with poor visibility in the MM5 UI.

Steps:
I tried downloading album "Don't Cry for No Hipster" by Ben Sidran. The batch ran, but did not complete.
I opened the Downloads node in the Media Tree, and saw that two tracks were suspended.
I resumed them, and only then got a error dbox saying that the two tracks had got "HTTP/1.1 403 Forbidden"
The tracks had been removed from my filtered grid, but not downloaded.
I re-opened Devices&Services>GPM>Alltracks, so that you would rebuild the list of my GPM contents.
And then re-requested that the tracks be downloaded.
This time the download completed OK, so the problem seems to have been recoverable.
This happened 3 times while testing today ... different tracks each time.

Suggestions:

If it is not possible to make a filtered grid more stable during download, it would be better if you cached grid updates while the download is in progress.
That way we could make fresh selections to add to the download batch, without MM5 looking broken.

Also the wheel should spin, at its normal position on the rhs side of the tool bar, while the download task is in progress.

You should improve handling of 403 error ... at the very least you could present the error dbox in real time, or somehow alert that the batch has some suspended items. At the moment this error indication is effectively hidden.

Top