Home Page
  • March 28, 2024, 09:33:06 am *
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

Official site launch very soon, hurrah!


Author Topic: Plex Playlist Importer  (Read 99790 times)

Skipper42

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Plex Playlist Importer
« Reply #30 on: June 13, 2016, 05:53:42 pm »

Well, I solved (kindasorta) the basic problem of BOM inclusion by installing EditPad Lite (freeware, unlike the Pro version), which permits me to create (Options menu) an M3U file type and specify UTF-8 and no BOM, and to replace the existing BOM with none if found. I can then open an .m3u created in another app (e.g., MediaMonkey, mp3tag, Notepad) and resave it with no BOM.
Unless there are character codes beyond the basic alphabet (see below for a description of this newly discovered problem), the .m3u will then import successfully...
...kindasorta. After the first two successful playlist imports, PMS stopped displaying the total duration (length) of the playlist, reporting only 'x Items' where x is the track count. Further, when the playlist is opened in Plex, all the track lengths are shown as 0:00. A quick check with mp3tag showed the metadata for length as indeed being included in the .mp3 file, and when one browses to the album in Plex, it, too, has the track length correctly shown. In both cases -- playlist or music browse -- Plex plays the track correctly. Interestingly, once a track has been played from the playlist, suddenly PMS finds its brain and correctly displays the track length on the Playlist page. There's no Refresh opportunity in the Playlist dialogue to force this otherwise. After the track lengths are displayed correctly once, they're displayed correctly henceforth, along with the overall playlist length on the main Playlist browse page. I noticed that it took a while for the data to propagate there, however. Even after playing all tracks in a 1:46 duration playlist (duration doesn't matter, simply starting each track is sufficient to update the track length display after a few seconds), the initial value shown for playlist length upon return to the main Playlist page was 20 minutes. Returning to the track listing for the playlist showed that indeed all the track lengths were still there, and on return up to the main Playlist page, suddenly the duration was the correct 1:46 value.
I'm unable from where I sit to determine whether this is a PMS bug or a PPI problem; perhaps there's a missing commit in the PPI SQL update query for the track metadata records, and absent some refresh mechanism, takes PMS a while to update itself?
Now on to the character set/codespace problem.
If the path (artist, album) or track filename contain other than basic alphanumeric characters thru ~ (code < 128 or 007F), then you run into problems like the following, where PPI can't find the track .mp3 to get its metadata:
Actual mp3 file record in m3u playlist:<path>\Music Library\Saint-Saëns\Violin Concerto No. 3\02 Romance for violin & orchestra in C major, Op 48.mp3
PPI import error:ListImporter 'Winamp playlist': Cannot find file listed in playlist (must be relative to the playlist):<path>\Music Library\Saint-Sa\xc3«ns\Violin Concerto No. 3\02 Romance for violin & orchestra in C major, Op 48.mp3
In this case, the e-umlaut (ë) has a decimal character code of 235 (+00EB).
Doesn't matter whether <path> in this example is relative ("dot notation") or absolute (<drive letter>:<folder path>). Just FYI - the error message always says "relative" even if absolute path addressing is used (likely an artifact of your absolute addressing quick fix).
Since PMS correctly renders the special characters and correctly finds and plays the track, and since other apps (MediaMonkey, mp3tag, Notepad, EditPad Lite) correctly find/render/play the .mp3 file, this flaw appears to be inside PPI.
I checked and it also doesn't matter whether the character set used is UTF-8 or Western European.
This would seem thus that PPI's character set issues are larger than just ignoring the UTF-8 BOM in an .m3u file.
I know that's not good news, but do hope there's enough info here for you to diagnose and correct the bug; I've made enough progress with a handful of successful playlist imports to be excited about the prospect of overall success!
Thanks again for all you've done on this.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #31 on: June 16, 2016, 07:13:05 pm »

Sorry for taking so long on this, should be able to look into this and update the software tomorrow or the day after.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #32 on: June 17, 2016, 10:03:25 pm »

OK, my forum has been randomly deleting posts  >:( I just recovered this from the trashbin.

Pierre: it requires python3. I will be making a minor change soon to verify it uses that.

ulle: Playlists are not user based, they are library based. If you mean you want to use a non-admin user to import a playlist, it is possible as long as they have access to the plex sqllite3 file.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #33 on: June 17, 2016, 10:05:54 pm »

OK, my forum is acting super crazy right now and not showing your last post >.<; here it is again, and I am working on the stuff now.

Skipper42:
Quote
Well, I solved (kindasorta) the basic problem of BOM inclusion by  installing EditPad Lite (freeware, unlike the Pro version), which  permits me to create (Options menu) an M3U file type and specify UTF-8  and no BOM, and to replace the existing BOM with none if found. I can  then open an .m3u created in another app (e.g., MediaMonkey, mp3tag,  Notepad) and resave it with no BOM.
Unless there are character codes  beyond the basic alphabet (see below for a description of this newly  discovered problem), the .m3u will then import successfully...
...kindasorta.  After the first two successful playlist imports, PMS stopped displaying  the total duration (length) of the playlist, reporting only 'x Items'  where x is the track count. Further, when the playlist is opened in  Plex, all the track lengths are shown as 0:00. A quick check with mp3tag  showed the metadata for length as indeed being included in the .mp3  file, and when one browses to the album in Plex, it, too, has the track  length correctly shown. In both cases -- playlist or music browse --  Plex plays the track correctly. Interestingly, once a track has been  played from the playlist, suddenly PMS finds its brain and correctly  displays the track length on the Playlist page. There's no Refresh  opportunity in the Playlist dialogue to force this otherwise. After the  track lengths are displayed correctly once, they're displayed correctly  henceforth, along with the overall playlist length on the main Playlist  browse page. I noticed that it took a while for the data to propagate  there, however. Even after playing all tracks in a 1:46 duration  playlist (duration doesn't matter, simply starting each track is  sufficient to update the track length display after a few seconds), the  initial value shown for playlist length upon return to the main Playlist  page was 20 minutes. Returning to the track listing for the playlist  showed that indeed all the track lengths were still there, and on return  up to the main Playlist page, suddenly the duration was the correct  1:46 value.
I'm unable from where I sit to determine whether this is a  PMS bug or a PPI problem; perhaps there's a missing commit in the PPI  SQL update query for the track metadata records, and absent some refresh  mechanism, takes PMS a while to update itself?
Now on to the character set/codespace problem.
If  the path (artist, album) or track filename contain other than basic  alphanumeric characters thru ~ (code < 128 or 007F), then you run  into problems like the following, where PPI can't find the track .mp3 to  get its metadata:
Actual mp3 file record in m3u  playlist:<path>\Music Library\Saint-Saëns\Violin Concerto No. 3\02  Romance for violin & orchestra in C major, Op 48.mp3
PPI import  error:ListImporter 'Winamp playlist': Cannot find file listed in  playlist (must be relative to the playlist):<path>\Music  Library\Saint-Sa\xc3«ns\Violin Concerto No. 3\02 Romance for violin  & orchestra in C major, Op 48.mp3
In this case, the e-umlaut (ë) has a decimal character code of 235 (+00EB).
Doesn't  matter whether <path> in this example is relative ("dot  notation") or absolute (<drive letter>:<folder path>). Just  FYI - the error message always says "relative" even if absolute path  addressing is used (likely an artifact of your absolute addressing quick  fix).
Since PMS correctly renders the special characters and  correctly finds and plays the track, and since other apps (MediaMonkey,  mp3tag, Notepad, EditPad Lite) correctly find/render/play the .mp3 file,  this flaw appears to be inside PPI.
I checked and it also doesn't matter whether the character set used is UTF-8 or Western European.
This would seem thus that PPI's character set issues are larger than just ignoring the UTF-8 BOM in an .m3u file.
I  know that's not good news, but do hope there's enough info here for you  to diagnose and correct the bug; I've made enough progress with a  handful of successful playlist imports to be excited about the prospect  of overall success!
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #34 on: June 18, 2016, 12:17:30 am »

OK, I merged in another authors branch to fix the linux absolute path problems. I fixed the UTF8 bom problem, so you don't have to worry about that anymore either. I also added cygwin compatibility.

I just tested with a utf8 m3u file and file names with Japanese characters and it imported fine (had to set up special test cases). Are you absolutely sure your file is utf8? Assuming your command line is in utf8, when the error is shown it should have the proper filename at the end of the error.

I also just tried to save a list from an older version of winamp and it just put question marks for any of the japanese characters ~.~

I also just did a massive import of all my music into plex (have been rebuilding my library). And interestingly enough, 0 of my 100+ files that had japanese characters even imported into plex!  >:( So are you also sure those files are actually in Plex?

There have been weird errors in the past regarding playlist and file lengths in plex. One example is that if your playlist has more than 150 entries, plex no longer recognizes it. The person on my forum that discovered it submitted a bug report to plex. Not sure if it was ever resolved.

But anywho, the playtimes are grabbed from what is already in the plex database, so it is possible it had not recorded them yet at the time you did your playlist import. Or perhaps they are storing additional information about the tracks elsewhere and not updating the main table. I didn't try very hard on that part, as I knew it would auto correct. Once the file is in the plex server playlist, my program no longer has any control over its values.

The error that you are showing regarding the charset looks like it might be b/c you are not in utf8 mode in your tty console. Also, the character you mentioned, c3, is "Ã", which is interesting.  I think I see that one a lot when there are encoding problems. If the board lets you, any chance you can post your playlist file with just that one song for me to inspect. And western european character set would definitely not work with the importer. UTF8 is essential.

Also, have you been running via the executable, or the python scripts? If the latter, I have the newest source up on github @ https://github.com/dakusan/PlexPlaylistImporter/ . If you are unable to run from source, I'll get an executable compiled for you (its a minor PITA). Not wanting to release a new version on the site until your problem is resolved
Logged

Skipper42

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Plex Playlist Importer
« Reply #35 on: June 20, 2016, 08:52:25 pm »


A. I'm as sure as I can be with the tools at my disposal that these .m3u's are UTF-8:
1. They're exports from Mp3tag, which allows me to establish the output character set in the .mte export spec (attached). It's set to UTF-8.
2. Notepad reports they're UTF-8.
3. I used EditPad Lite to force the text file to be no-BOM, UTF-8.

B. I'm certain the underlying .mp3 files are present in both the library at the location found and are individually playable in Plex (and thus in the Plex Library). The .mp3 files appear in all Plex Library listings irrespective of filtering (Album, Artist...).

C. I'm using your Windows .exe only; I do not have Python running separately. I'm running in a Win 7 x64 environment, in case that matters; machine resources should not be an issue (quad-core i7, 32GB RAM).

D. The longest playlist I've tried to import (5.5hr running time) has only 73 tracks, but I appreciate the heads-up on the 150 track limit.

E. Also attached per your request is a fragment of the playlist with the problematic character set entry described above.

Hope this helps. Thanks again very much for your efforts.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #36 on: June 25, 2016, 03:24:48 am »

I'm working on updating the software this weekend.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #37 on: July 01, 2016, 04:16:31 am »

I have made multiple updates to the software and recompiled it. I have attached it to this post if you want to check it out. I confirmed that it worked fine in command prompt with your m3u file. Update log is on github.

P.S. If possible, please let me know within the next few days if all is well now. Would like to get the confirmed updates and compilation up on my website.
« Last Edit: July 01, 2016, 04:25:58 am by Dakusan »
Logged

Skipper42

  • Jr. Member
  • **
  • Posts: 9
    • View Profile
Re: Plex Playlist Importer
« Reply #38 on: July 04, 2016, 03:15:18 pm »

The latest ListImporter version for Windows works like a champ - thanks!I've successfully tested with:
  • Both relative and absolute paths to track files in the m3u
  • With m3u's exported from MediaMonkey (aka MM; ver 4.1.12.1798; I'll let you know if this changes with the latest version, 4.1.13.1801)
  • With m3u's exported from Mp3tag, and
  • With m3u's exported from my iBasso DX90.
I confirmed that byte order no longer matters; I no longer have to go thru the extra pass through EditPad Lite to drop the BOM.If an actual file cannot be found from its m3u reference, I've confirmed that ListImporter correctly notifies the user of the path and filename of the file it cannot find (with one exception; please see below).If errors are encountered, ListImporter correctly sets Errorlevel so it can be checked in a batch file. Thus, I can use this to avoid prematurely archiving failed import playlists until the issues are resolved.I've not found a good way to execute ListImporter from a folder not containing both the .exe and all its .dll files. This would be useful but is not essential. I do have to manage Acronis' insistence on restarting one of its supposedly unneeded services that uses a different sqlite3.dll. The sqlite3.dll used by the latest version of PMS no longer conflicts.I did find a couple of small residual issues plus a couple of what you're free to view as extremely minor quibbles.The first of two small issues is that ListImporter is intolerant of blank lines in the m3u (e.g., at the end). This leads to the mysterious error message that the track file must be located relative to the playlist, but since the reference line is a blank, there's no file reported. Took some lengthy trial and error to find what was causing the glitch. I'll note that part of my confusion was that the same error message ("relative to") appeared irrespective of whether I was using dot notation relative paths or absolute paths for the track references (I suspect this is a holdover from versions prior to those permitting absolute path references).The second small issue is that any error in one track reference causes the entire import to abort with Errorlevel set to 1. That meant the original m3u with one or more blank lines would abort, while trying to identify the problem track reference by successive runs each with a new reference added from the original always was successful. That's actually how I tumbled to the blank line intolerance.My benchmark for whether the m3u had valid references has been Florian Heidenreich's excellent freeware Mp3tag (http://www.mp3tag.de/en/). In sum, if Mp3tag can import all the rows successfully, then I can safely assume the m3u is valid; it reports any track files it cannot find. That means anyone should be able to debug issues with ListImporter using Mp3tag to test the m3u's track file references for validity (plus verifying that the track files can be played independently in Plex).I also use Mp3tag to convert relative paths in m3u's to absolute or relative paths matching the location of the m3u file to be imported to Plex. MM's relative paths are relative to where the m3u is originally stored). MM is a far richer playlist editor/maker than Plex, so combining your tool with my methods gives me the best of both worlds.The .mte export configuration file I use in Mp3tag is:$filename(<path>\to_import\Plex.m3u,utf-8)$loop(%_counter%)%_folderpath%%_filename_ext%$loopend()...with care to have the file end after the $loopend() statement so as not to generate the blank line that ListImporter thinks is an invalid track reference. In my case, E:\Users\Public\Music\My Playlists\! Plex Playlists\ is the folder also containing the ListImporter .exe, .dll's and ppibatch.bat file.Once exported, I rename Plex.m3u to whatever I want the playlist name to be. Note: to export with relative vs. absolute paths, change %_folderpath% to %_folderpath_rel%, tho ListImporter always worked irrespective of which was used.The two minor quibbles are (a) execution message output appears to go to stderr vs. stdout, which can interfere with batch file output piping for logging/analysis purposes, and (b) delays/problems updating the track length once the playlist has been imported into Plex (an issue I noted in a previous post).The batch file I use (ppibatch.bat, attached) presumes the m3u playlists to be imported are in a particular 'to_import' folder, and if the import of each is successful, the m3u file is moved to an 'imported' folder in the same hierarchy.In the batch file, I used UTF-8 encoding, but set it explicitly in the command line vs. relying on the default. Ditto the file type setting (explicitly 'mp3' vs. default). These were more for self-documentation than any requirement to do so; worked fine with the defaults. Ditto the location of the Plex database.When piped to a log file, each execution in my batch file generates an entry of the following form:Sat 07/02/2016 14:38:28.48PlexPlaylistImporter.exe -f -e utf-8 -t m3u "<path>\to_import\5hr Playlist.m3u" "5hr Playlist"73 items imported        1 file(s) moved....showing there could be value in having error messages, etc. in the consolidated app+OS output also.The track length update problem seems to be avoidable if one turns on frequent Library updating and updating of the Library with any change to the Library, prior to importing the m3u. If that's not done, the track lengths of some tracks remain at 0:00 after import. This also means the overall playlist length will be misreported until each of the 0:00 tracks is launched and another track launched afterward, at which point Plex updates the playlist track length entry and overall playlist length. Even forcing Library updates failed to update the track and playlist lengths.I cannot tell from info available to me here whether the track length update problem is a Plex bug/feature (they clearly have yet to envision importing m3u playlists, and thus without your tool would have no way to test for this bug!) or whether there's some other additional Plex database field references that need updating in your update query SQL (e.g., possibley a field in a different table from the main Playlist records). Regardless, once I turned on frequent Library updating/updating after every change, the problem seemed to disappear.With all that in mind, and given the general end-to-end utility now available, I recommend you post an update to the Plex Forum that details these working methods. Please feel free to share my Mp3tag export .mte above and batch import method's .bat file attached in your post.Thanks again very much for all your efforts here. This has now made Plex finally usable for me!
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #39 on: July 05, 2016, 01:53:02 am »

Yikes. It feels like on a lot of this you are going to way more trouble than should be needed :-\

Quote
I've not found a good way to execute ListImporter from a folder not containing both the .exe and all its .dll files. This would be useful but is not essential. I do have to manage Acronis' insistence on restarting one of its supposedly unneeded services that uses a different sqlite3.dll. The sqlite3.dll used by the latest version of PMS no longer conflicts.
I think I might actually have been providing a bad sqlite3.dll with my previous releases, which could have been the problem. That is fixed in the new archive I had sent you. I don't really see an easier way to do distribute this without the dlls as separate files in the archive. I'm not big on installers.
Quote
The first of two small issues is that ListImporter is intolerant of blank lines in the m3u (e.g., at the end).
I have updated the code to ignore lines that are blank or only have whitespace. It is up on github and I will included it on my next executable install.
Quote
The second small issue is that any error in one track reference causes the entire import to abort with Errorlevel set to 1.  [...]
This is always something I haven't really been sure how to handle. Do you have any suggestions? I guess I could include a batch error mode that would be the default...
Quote
(a) execution message output appears to go to stderr vs. stdout, which can interfere with batch file output piping for logging/analysis purposes
I believe throwing errors to stderr is the proper behavior. Would it help if I included a parameter flag that instead pushed everything through stdout? In bash shell environments, it's as simple as adding "2>&1" after your command. Not sure about windows.
I think that covers everything you mentioned, minus track length. I just looked in the DB, and from a quick glance, I'm not actually sure where the track durations are stored for items on a playlist. I think I skipped that since I wanted to make a minimum number of modifications to the database and knew it would auto correct.

[Edit] P.S. The latest version (have not compiled into exe yet) allows dragging playlists onto the executable.
« Last Edit: July 05, 2016, 02:01:34 am by Dakusan »
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #40 on: July 26, 2016, 08:12:09 pm »

You mean a separate Plex pass user, correct? One in which you would have had to of shared your libraries with via Libraries->...->Share ? (Or shares visible via plex.tv->launch->settings->users->friends)
« Last Edit: July 26, 2016, 08:15:58 pm by Dakusan »
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #41 on: February 14, 2017, 10:23:31 am »

In PlexPlaylistImporter.py, below line 177
Code: [Select]
#Insert the items into the playlistAdd the following line
Code: [Select]
Cur.execute('DELETE FROM play_queue_generators WHERE playlist_id=?', (PlexPlaylistID,))
I have not tested this code, so make sure to backup your database before attempting it.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #42 on: February 17, 2017, 02:47:07 am »

Same disclaimer. Guess I gotta add command line flags for this and update on github. Already have 1 pull request on github. PITA cause I don't release code without thoroughly testing.

Line 137. Change
Code: [Select]
sys.exit("File not found in DB: "+FilePath)to
Code: [Select]
continueWhen I update on github it will throw a warning instead.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #43 on: June 19, 2017, 04:09:42 pm »

"DB Error: no such module: fts4" indicates an outdated or incomplete sqlite3 version. Google also points out that this happens when trying to compile for iOS, or possibly osx.
Logged

Dakusan

  • Programmer Person
  • Administrator
  • Hero Member
  • *****
  • Posts: 535
    • View Profile
    • Dakusan's Domain
Re: Plex Playlist Importer
« Reply #44 on: June 24, 2017, 11:47:49 am »

First, "/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db" needs to be quoted, since it has spaces, or your command line interpreter will think those are separate parameters.

I'm surprised it decided to import at all with what you gave it, since you gave an unquoted (and therefore bogus) path to the database. I'm guessing it found the path on its own so it didn't error out.

Second, you only gave the "Playlist_Path" parameter and not a "Plex_Playlist_Name" parameter.
« Last Edit: June 24, 2017, 11:50:47 am by Dakusan »
Logged