I wrote the code at the end of this post to test the different forms of string concatenation and they really are all almost exactly equal in both memory and time footprints.
The two primary methods I used are concatenating strings onto each other, and filling an array with strings and then imploding them. I did 500 string additions with a 1MB string in PHP 5.6 (so the result is a 500MB string). At every iteration of the test, all memory and time footprints were very very close (at ~$IterationNumber*1MB). The runtime of both tests was 50.398 seconds and 50.843 seconds consecutively which are most likely within acceptable margins of error.
Garbage collection of strings that are no longer referenced seems to be pretty immediate, even without ever leaving the scope. Since the strings are mutable, no extra memory is really required after the fact.
HOWEVER, The following tests showed that there is a different in peak memory usage WHILE the strings are being concatenated.
Result=6,613,320 bytes (~6MB w/o the initial PHP memory footprint)
So there is in fact a difference that could be significant in very very large string concatenations memory-wise (I have run into such examples when creating very large data sets or SQL queries).
But even this fact is disputable depending upon the data. For example, concatenating 1 character onto a string to get 50 million bytes (so 50 million iterations) took a maximum amount of 50,322,512 bytes (~48MB) in 5.97 seconds. While doing the array method ended up using 7,337,107,176 bytes (~6.8GB) to create the array in 12.1 seconds, and then took an extra 4.32 seconds to combine the strings from the array.
Anywho... the below is the benchmark code I mentioned at the beginning which shows the methods are pretty much equal. It outputs a pretty HTML table.
<? //Please note, for the recursion test to go beyond 256, xdebug.max_nesting_level needs to be raised. //You also may need to update your memory_limit depending on the number of iterations
//Output the start memory print 'Start: '.memory_get_usage()."B
Below test results are in MB ";
//Our 1MB string global $OneMB, $NumIterations; $OneMB=str_repeat('x', 1024*1024); $NumIterations=500;
//Run the tests $ConcatTest=RunTest('ConcatTest'); $ImplodeTest=RunTest('ImplodeTest'); $RecurseTest=RunTest('RecurseTest');
//Output the results in a table OutputResults( Array('ConcatTest', 'ImplodeTest', 'RecurseTest'), Array($ConcatTest, $ImplodeTest, $RecurseTest) );
//Start a test run by initializing the array that will hold the results and manipulating those results after the test is complete function RunTest($TestName) { $CurrentTestNums=Array(); $TestStartMem=memory_get_usage(); $StartTime=microtime(true); RunTestReal($TestName, $CurrentTestNums, $StrLen); $CurrentTestNums[]=memory_get_usage();
//Subtract $TestStartMem from all other numbers foreach($CurrentTestNums as &$Num) $Num-=$TestStartMem; unset($Num);
//Initialize the test and store the memory allocated at the end of the test, with the result function RunTestReal($TestName, &$CurrentTestNums, &$StrLen) { $R=$TestName($CurrentTestNums); $CurrentTestNums[]=memory_get_usage(); $StrLen=strlen($R); }
//Concatenate 1MB string over and over onto a single string function ConcatTest(&$CurrentTestNums) { global $OneMB, $NumIterations; $Result=''; for($i=0;$i<$NumIterations;$i++) { $Result.=$OneMB; $CurrentTestNums[]=memory_get_usage(); } return $Result; }
//Create an array of 1MB strings and then join w/ an implode function ImplodeTest(&$CurrentTestNums) { global $OneMB, $NumIterations; $Result=Array(); for($i=0;$i<$NumIterations;$i++) { $Result[]=$OneMB; $CurrentTestNums[]=memory_get_usage(); } return implode('', $Result); }
//Recursively add strings onto each other function RecurseTest(&$CurrentTestNums, $TestNum=0) { Global $OneMB, $NumIterations; if($TestNum==$NumIterations) return '';
Additional string parameters that have %THEVAR% are replaced with the value being checked
If the first item is NULL, it will be removed, and the return will include full rows ("*") of the query result set
Updated functionality:
json_encode() In RetMsg() and CallByAction() now use JSON_UNESCAPED_UNICODE
GetVars.VarArray.IsOptional now only triggers if it is (boolean)true
Added additional description specification to GetVars.VarArray.SQLLookup which says it uses the additional parameters as values to fill in the SQL Query
Added static member $InitialPrintAndDieOnError which DSQL.PrintAndDieOnError inherits on creation
Bug Fixes:
mysqli_set_charset is set to utf-8
In FormatSQLError() the date() function used for “Start Time” now uses “24-hour format of an hour with leading zeros” [date(“H”)] instead of “12-hour format of an hour without leading zeros” [date(“g”)]
Following is some C++ source code for a Windows kernel-driver service loader. It could be used to load other service types too by changing the dwServiceType flag on the CreateService call. I threw this together for another project I am currently working on. It is also used in the following post (posting soon).
It works in the following way:
It is a command line utility which takes 3 arguments:
The service name. Hereby referred to as SERVICE_NAME
The service display name. Hereby referred to as DISPLAY_NAME
The driver path (to the .sys file). Hereby referred to as DRIVER_PATH
This program (most likely) requires administrative access. There are also some caveats regarding driver code signing requirements that are thoroughly explored elsewhere.
It first checks to see if a service already exists with the given SERVICE_NAME. If it does:
If the DISPLAY_NAME matches, the service is kept as is.
If the DISPLAY_NAME does not match, the user is prompted on if they want to delete the current service. If they do not, the program exits.
If the service needs to be created (it did not already exist or was deleted), it creates the service with the given SERVICE_NAME, DISPLAY_NAME, and DRIVER_PATH. If the service is not created during this run, the DRIVER_PATH is ignored. Note: The DRIVER_PATH must be to a direct local file system file. I have found that network links and symbolic links do not work.
The service is started up:
If it is already running, the user is prompted on if they want to stop the currently running service. If they say no, the program exits.
The program then waits for a final user input on if they want to close the service before exiting the program.
If there was an error, the program reports the error, otherwise, it reports “Success”.
The program pauses at the end until the user presses any key to exit.
The program returns 0 on success, and 1 if an error occurred.
//Compiler flags #define WIN32_LEAN_AND_MEAN //Include minimum amount of windows stuff #ifndef _UNICODE //Everything in this script is unicode #define _UNICODE #endif
//Function declarations WCHAR* InitDriver(int argc, WCHAR *argv[]); WCHAR* FormatError(WCHAR* Format, ...); SmartWinAlloc GetLastErrorStr(); BOOLEAN AskQuestion(WCHAR* Question); //Returns if user answered yes
int wmain(int argc, WCHAR *argv[]) { //Run the init routine WCHAR* Ret=InitDriver(argc, argv);
//If there is an error, report it, or otherwise, report success wprintf(L"%s\n", Ret ? Ret : L"Success"); wprintf(L"%s\n", L"Press any key to exit"); _getch();
//Open the service manager wprintf(L"%s\n", L"Opening the service manager"); SC_HANDLE HSCManager=OpenSCManager(nullptr, nullptr, SC_MANAGER_CREATE_SERVICE); if(!HSCManager) return FormatError(L"%s: %s", L"Error opening service manager", GetLastErrorStr()); SmartCloseService FreeHSCManager(&HSCManager, Delete_SmartCloseService);
//Check if the service already exists wprintf(L"%s\n", L"Checking previously existing service state"); BOOL ServiceExists=false; { //Get the service name const DWORD NameBufferSize=255; WCHAR NameBuffer[NameBufferSize]; WCHAR *NamePointer=NameBuffer; DWORD NamePointerSize=NameBufferSize; std::unique_ptr<WCHAR> Buf(nullptr); //May be swapped with a real pointer later for(INT_PTR i=0;i<2;i++) { //If we found the service, exit the lookup here if(GetServiceDisplayName(HSCManager, Param_ServiceName, NamePointer, &NamePointerSize)) { ServiceExists=true; break; }
//If the service does not exist, we can exit the lookup here if(GetLastError()==ERROR_SERVICE_DOES_NOT_EXIST) break;
//If error is not insufficient buffer size, return the error if(GetLastError()!=ERROR_INSUFFICIENT_BUFFER) return FormatError(L"%s: %s", L"Could not query service information", GetLastErrorStr());
//If second pass, error out if(i==1) return FormatError(L"%s: %s", L"Could not query service information", L"Second buffer pass failed");
//Create a buffer of appropriate size (and make sure it will later be released) NamePointer=new WCHAR[++NamePointerSize]; std::unique_ptr<WCHAR> Buf2(NamePointer); Buf.swap(Buf2); }
//If the service already exists, confirm the service name matches, and if not, ask if user wants to delete the current service if(ServiceExists) { wprintf(L"%s\n", L"The service already exists"); if(wcsncmp(NamePointer, Param_DisplayName, NamePointerSize+1)) { //If the server names do not match, ask the user what to do wprintf(L"%s:\nCurrent: %s\nRequested: %s\n", L"The service names do not match", NamePointer, Param_DisplayName);
//Make the request if(!AskQuestion(L"Would you like to replace the service? (y/n)")) //If user does not wish to replace the service return FormatError(L"%s", L"Cannot continue if service names do not match");
//Delete the service wprintf(L"%s\n", L"Deleting the old service"); ServiceExists=false; SC_HANDLE TheService=OpenService(HSCManager, Param_ServiceName, DELETE); if(!TheService) return FormatError(L"%s: %s", L"Could not open the service to delete it", GetLastErrorStr()); SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service handle if(!DeleteService(TheService)) return FormatError(L"%s: %s", L"Could not delete the service", GetLastErrorStr()); wprintf(L"%s\n", L"The service has been deleted"); } } }
//Create the service SC_HANDLE TheService; if(!ServiceExists) { //Confirm the driver path exists wprintf(L"%s\n", L"Checking the driver file"); DWORD FileAttrs=GetFileAttributes(Param_DriverPath); if(FileAttrs==INVALID_FILE_ATTRIBUTES) return FormatError(L"%s: %s", L"Given path is invalid", GetLastErrorStr()); if(FileAttrs&FILE_ATTRIBUTE_DIRECTORY) return FormatError(L"%s: %s", L"Given path is invalid", L"Path is a folder");
//Create the service wprintf(L"%s\n", L"Creating the service"); TheService=CreateService( HSCManager, Param_ServiceName, Param_DisplayName, SERVICE_START|SERVICE_STOP, SERVICE_KERNEL_DRIVER, SERVICE_DEMAND_START, SERVICE_ERROR_IGNORE, Param_DriverPath, nullptr, nullptr, nullptr, nullptr, nullptr); if(!TheService) return FormatError(L"%s: %s", L"Could not create the service", GetLastErrorStr());
//Open the service if not creating } else { TheService=OpenService(HSCManager, Param_ServiceName, SERVICE_START|SERVICE_STOP); if(!TheService) return FormatError(L"%s: %s", L"Could not open the service", GetLastErrorStr()); } SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service on exit
//Start the service wprintf(L"%s\n", L"Starting the service"); for(INT_PTR i=0;i<2;i++) { if(StartService(TheService, 0, nullptr)) break;
//If not "service already running" error, or user does not want to stop the current service if(i==1 || GetLastError()!=ERROR_SERVICE_ALREADY_RUNNING || !AskQuestion(L"The service is already running. Would you like to stop it? (y/n)")) return FormatError(L"%s: %s", L"Could not start the service", GetLastErrorStr());
//Stop the service SERVICE_STATUS ss; wprintf(L"%s\n", L"Stopping the current service"); if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss)) return FormatError(L"%s: %s", L"Could not stop the current service", GetLastErrorStr()); } wprintf(L"%s\n", L"Started the service");
//Ask if the user wants to close the service if(!AskQuestion(L"Would you like to stop the service before exit? (y/n)")) return nullptr;
//Stop the service SERVICE_STATUS ss; if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss)) return FormatError(L"%s: %s", L"Could not stop the service", GetLastErrorStr()); if(ss.dwCurrentState!=SERVICE_STOP_PENDING && ss.dwCurrentState!=SERVICE_STOPPED) return FormatError(L"%s", L"The service does not appear to be closing"); wprintf(L"%s\n", L"The service has been stopped");
//Return success return nullptr; }
WCHAR* FormatError(WCHAR* Format, ...) { static WCHAR Err[255]; va_list VAList; va_start(VAList, Format); vswprintf(Err, sizeof(Err)/sizeof(Err[0]), Format, VAList); return Err; }
BOOLEAN AskQuestion(WCHAR* Question) { //Make the request and wait for an input character while(1) { //Ask the question and get the answer wprintf(L"%s:", Question); fflush(stdout); char InputChar=_getch(); printf("\n");
//Check for a valid answer if(InputChar=='n' || InputChar=='N') return FALSE; if(InputChar=='y' || InputChar=='Y') return TRUE; } }
You mean a separate Plex pass user, correct? One in which you would have had to of shared your libraries with via Libraries->...->Share ? (Or shares visible via plex.tv->launch->settings->users->friends)
Fixed absolute path logic for Linux (partial credit to Matt Spitz)
Bypasses UTF8 BOM
Program now works off of argument flags.
Added parameters: Playlist encoding, override type, force list
Created special BulletHelpFormatter class for parameters
Playlist names can now conflict with other item/list names in Plex
Console column width passthrough in the .sh file
Updated READMEs regarding:
Unicode compliance
The “no such module : FTS4” error
Running the script from a computer external to the server running Plex
The “The program can’t start because MSVCR100.dll is missing” error
All updates
Added the ability to compile to a windows executable (via setup.py py2exe)
The Playlist Name is now an optional argument which can be entered after the program is ran. This allows directly dragging playlists onto the executable
Added shebang to main script
M3U files now ignore lines that are empty or have only whitespace
Yikes. It feels like on a lot of this you are going to way more trouble than should be needed
Quote
I've not found a good way to execute ListImporter from a folder not containing both the .exe and all its .dll files. This would be useful but is not essential. I do have to manage Acronis' insistence on restarting one of its supposedly unneeded services that uses a different sqlite3.dll. The sqlite3.dll used by the latest version of PMS no longer conflicts.
I think I might actually have been providing a bad sqlite3.dll with my previous releases, which could have been the problem. That is fixed in the new archive I had sent you. I don't really see an easier way to do distribute this without the dlls as separate files in the archive. I'm not big on installers.
Quote
The first of two small issues is that ListImporter is intolerant of blank lines in the m3u (e.g., at the end).
I have updated the code to ignore lines that are blank or only have whitespace. It is up on github and I will included it on my next executable install.
Quote
The second small issue is that any error in one track reference causes the entire import to abort with Errorlevel set to 1. [...]
This is always something I haven't really been sure how to handle. Do you have any suggestions? I guess I could include a batch error mode that would be the default...
Quote
(a) execution message output appears to go to stderr vs. stdout, which can interfere with batch file output piping for logging/analysis purposes
I believe throwing errors to stderr is the proper behavior. Would it help if I included a parameter flag that instead pushed everything through stdout? In bash shell environments, it's as simple as adding "2>&1" after your command. Not sure about windows. I think that covers everything you mentioned, minus track length. I just looked in the DB, and from a quick glance, I'm not actually sure where the track durations are stored for items on a playlist. I think I skipped that since I wanted to make a minimum number of modifications to the database and knew it would auto correct.
[Edit] P.S. The latest version (have not compiled into exe yet) allows dragging playlists onto the executable.
The PHP MySQL extension is being deprecated in favor of the MySQLi extension in PHP 5.5, and removed as of PHP 7.0. MySQLi was first referenced in PHP v5.0.0 beta 4 on 2004-02-12, with the first stable release in PHP 5.0.0 on 2004-07-13[1]. Before that, the PHP MySQL extension was by far the most popular way of interacting with MySQL on PHP, and still was for a very long time after. This website was opened only 2 years after the first stable release!
With the deprecation, problems from some websites I help host have popped up, many of these sites being very, very old. I needed a quick and dirty solution to monkey-patch these websites to use MySQLi without rewriting all their code. The obvious answer is to overwrite the functions with wrappers for MySQLi. The generally known way of doing this is with the Advanced PHP Debugger (APD). However, using this extension has a lot of requirements that are not appropriate for a production web server. Fortunately, another extension I recently learned of offers the renaming functionality; runkit. It was a super simple install for me.
From the command line, run “pecl install runkit”
Add “extension=runkit.so” and “runkit.internal_override=On” to the php.ini
Besides the ability to override these functions with wrappers, I also needed a way to make sure this file was always loaded before all other PHP files. The simple solution for that is adding “auto_prepend_file=/PATH/TO/FILE” to the “.user.ini” in the user’s root web directory.
The code for this script is as follows. It only contains a limited set of the MySQL functions, including some very esoteric ones that the web site used. This is not a foolproof script, but it gets the job done.
//If a connection is not explicitely passed to a mysql_ function, use the last created connection global$SQLLink; //The remembered SQL Link functionGetConn($PassedConn) { if(isset($PassedConn)) return$PassedConn; global$SQLLink; return$SQLLink; }
def_split_lines(self, text, width): #Split lines around line breaks and then modify each line Lines=[] for Line in text.splitlines(): #Get the number of spaces to put at subsequent lines #0 if not a list item, oherwise, 2+list item start ListEl=re.match(r'^*\*', Line) NumBeginningSpace=(0if ListEl==Noneelse ListEl.end()+1)
#Add extra spaces at the beginning of each line to match the start of the current line, and go to a maxium of $width IsFirstPass=True SpacesToAdd='' NumSpacesToAdd=0 while(True): #Get the word break points before and after where the line would end MaxLineLen=max(min(width-NumSpacesToAdd, len(Line)), 1) PrevWordBreak=CurWordBreak=0 for WordBreak in re.finditer(r'(?<=\W).|\W|$', Line): PrevWordBreak=CurWordBreak CurWordBreak=WordBreak.start() if CurWordBreak>=MaxLineLen: if CurWordBreak==MaxLineLen: PrevWordBreak=CurWordBreak break
#If previous wordbreak is more than MinCharsInSplitWord away from MaxLineLen, then split at the end of the line IsSplit=(PrevWordBreak<1or CurWordBreak-PrevWordBreak>self.MinCharsInSplitWord) SplitPos=(MaxLineLen if IsSplit else PrevWordBreak)
#Append the new line to the list of lines Lines.append(SpacesToAdd+Line[0:SplitPos]+('-'if IsSplit else'')) Line=Line[SplitPos:]
#If this is the end, nothing left to do iflen(Line)==0: break
#If this is the first pass, update line creation variables if IsFirstPass: IsFirstPass=False NumSpacesToAdd=NumBeginningSpace SpacesToAdd=(''* NumSpacesToAdd)
I have made multiple updates to the software and recompiled it. I have attached it to this post if you want to check it out. I confirmed that it worked fine in command prompt with your m3u file. Update log is on github.
P.S. If possible, please let me know within the next few days if all is well now. Would like to get the confirmed updates and compilation up on my website.
I guess that french characters is the cause of that error:"List Importer 'Winamp playlist':'utf-8' codec can't decode byte 0xe9 in position 323:invalid continuarion byte
I have added a command line flag that lets you set the encoding of the file. You will want to set it to ISO-8859-1.
Quote
I chose an full English playlist. I have that:"Playlist name has already been taken by a non-playlist"
This has been fixed.
Quote
It seem that my plex playlist name can't be the same as the artist that is already in my plex library, so I change the playlist name to "paramorelist":DB Error: no such module fts4
I have added the following to the README file:
Quote
If running this mentions something about “no such module : FTS4”, you may need to replace the sqlite3.dll or sqlite3.so for your Python, which can be found at https://www.sqlite.org/download.html . For a Python for Windows install, the DLL location will most likely be located at C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python%PYTHON_VERSION%\DLLs.
The new version is available on github. Will be uploading a new windows executable within the next few days.