Home Page
  • July 25, 2025, 10:39:52 am *
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

Official site launch very soon, hurrah!


Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Dakusan

Pages: 1 ... 3 4 [5] 6 7 ... 38
61
Posts / PHP String Concatenation - Stringbuilder results
« on: September 26, 2016, 12:37:37 am »

I wrote the code at the end of this post to test the different forms of string concatenation and they really are all almost exactly equal in both memory and time footprints.


The two primary methods I used are concatenating strings onto each other, and filling an array with strings and then imploding them. I did 500 string additions with a 1MB string in PHP 5.6 (so the result is a 500MB string). At every iteration of the test, all memory and time footprints were very very close (at ~$IterationNumber*1MB). The runtime of both tests was 50.398 seconds and 50.843 seconds consecutively which are most likely within acceptable margins of error.

Garbage collection of strings that are no longer referenced seems to be pretty immediate, even without ever leaving the scope. Since the strings are mutable, no extra memory is really required after the fact.


HOWEVER, The following tests showed that there is a different in peak memory usage WHILE the strings are being concatenated.



$OneMB=str_repeat('x', 1024*1024);
$Final=$OneMB.$OneMB.$OneMB.$OneMB.$OneMB;
print memory_get_peak_usage();
Result=10,806,800 bytes (~10MB w/o the initial PHP memory footprint)


$OneMB=str_repeat('x', 1024*1024);
$Final=implode('', Array($OneMB, $OneMB, $OneMB, $OneMB, $OneMB));
print memory_get_peak_usage();
Result=6,613,320 bytes (~6MB w/o the initial PHP memory footprint)

So there is in fact a difference that could be significant in very very large string concatenations memory-wise (I have run into such examples when creating very large data sets or SQL queries).

But even this fact is disputable depending upon the data. For example, concatenating 1 character onto a string to get 50 million bytes (so 50 million iterations) took a maximum amount of 50,322,512 bytes (~48MB) in 5.97 seconds. While doing the array method ended up using 7,337,107,176 bytes (~6.8GB) to create the array in 12.1 seconds, and then took an extra 4.32 seconds to combine the strings from the array.


Anywho... the below is the benchmark code I mentioned at the beginning which shows the methods are pretty much equal. It outputs a pretty HTML table.


<?
//Please note, for the recursion test to go beyond 256, xdebug.max_nesting_level needs to be raised.
//You also may need to update your memory_limit depending on the number of iterations

//Output the start memory
print 'Start: '.memory_get_usage()."B

Below test results are in MB
";

//Our 1MB string
global $OneMB, $NumIterations;
$OneMB=str_repeat('x', 1024*1024);
$NumIterations=500;

//Run the tests
$ConcatTest=RunTest('ConcatTest');
$ImplodeTest=RunTest('ImplodeTest');
$RecurseTest=RunTest('RecurseTest');

//Output the results in a table
OutputResults(
 Array('ConcatTest', 'ImplodeTest', 'RecurseTest'),
 Array($ConcatTest, $ImplodeTest, $RecurseTest)
);

//Start a test run by initializing the array that will hold the results and manipulating those results after the test is complete
function RunTest($TestName)
{
 $CurrentTestNums=Array();
 $TestStartMem=memory_get_usage();
 $StartTime=microtime(true);
 RunTestReal($TestName, $CurrentTestNums, $StrLen);
 $CurrentTestNums[]=memory_get_usage();

 //Subtract $TestStartMem from all other numbers
 foreach($CurrentTestNums as &$Num)
   $Num-=$TestStartMem;
 unset($Num);

 $CurrentTestNums[]=$StrLen;
 $CurrentTestNums[]=microtime(true)-$StartTime;

 return $CurrentTestNums;
}

//Initialize the test and store the memory allocated at the end of the test, with the result
function RunTestReal($TestName, &$CurrentTestNums, &$StrLen)
{
 $R=$TestName($CurrentTestNums);
 $CurrentTestNums[]=memory_get_usage();
 $StrLen=strlen($R);
}

//Concatenate 1MB string over and over onto a single string
function ConcatTest(&$CurrentTestNums)
{
 global $OneMB, $NumIterations;
 $Result='';
 for($i=0;$i<$NumIterations;$i++)
 {
   $Result.=$OneMB;
   $CurrentTestNums[]=memory_get_usage();
 }
 return $Result;
}

//Create an array of 1MB strings and then join w/ an implode
function ImplodeTest(&$CurrentTestNums)
{
 global $OneMB, $NumIterations;
 $Result=Array();
 for($i=0;$i<$NumIterations;$i++)
 {
   $Result[]=$OneMB;
   $CurrentTestNums[]=memory_get_usage();
 }
 return implode('', $Result);
}

//Recursively add strings onto each other
function RecurseTest(&$CurrentTestNums, $TestNum=0)
{
 Global $OneMB, $NumIterations;
 if($TestNum==$NumIterations)
   return '';

 $NewStr=RecurseTest($CurrentTestNums, $TestNum+1).$OneMB;
 $CurrentTestNums[]=memory_get_usage();
 return $NewStr;
}

//Output the results in a table
function OutputResults($TestNames, $TestResults)
{
 global $NumIterations;
 print '<table border=1 cellspacing=0 cellpadding=2><tr><th>Test Name</th><th>'.implode('</th><th>', $TestNames).'</th></tr>';
 $FinalNames=Array('Final Result', 'Clean');
 for($i=0;$i<$NumIterations+2;$i++)
 {
   $TestName=($i<$NumIterations ? $i : $FinalNames[$i-$NumIterations]);
   print "<tr><th>$TestName</th>";
   foreach($TestResults as $TR)
     printf('<td>%07.4f</td>', $TR[$i]/1024/1024);
   print '</tr>';
 }

 //Other result numbers
 print '<tr><th>Final String Size</th>';
 foreach($TestResults as $TR)
   printf('<td>%d</td>', $TR[$NumIterations+2]);
 print '</tr><tr><th>Runtime</th>';
   foreach($TestResults as $TR)
     printf('<td>%s</td>', $TR[$NumIterations+3]);
 print '</tr></table>';
}
?>

62
Posts / Deep object compare for javascript
« on: September 24, 2016, 02:40:27 am »


function DeepObjectCompare(O1, O2)
{
   try {
      DOC_Val(O1, O2, ['O1->O2', O1, O2]);
      return DOC_Val(O2, O1, ['O2->O1', O1, O2]);
   } catch(e) {
      console.log(e.Chain);
      throw(e);
   }
}
function DOC_Error(Reason, Chain, Val1, Val2)
{
   this.Reason=Reason;
   this.Chain=Chain;
   this.Val1=Val1;
   this.Val2=Val2;
}

function DOC_Val(Val1, Val2, Chain)
{
   function DoThrow(Reason, NewChain) { throw(new DOC_Error(Reason, NewChain!==undefined ? NewChain : Chain, Val1, Val2)); }

   if(typeof(Val1)!==typeof(Val2))
      return DoThrow('Type Mismatch');
   if(Val1===null || Val1===undefined)
      return Val1!==Val2 ? DoThrow('Null/undefined mismatch') : true;
   if(Val1.constructor!==Val2.constructor)
      return DoThrow('Constructor mismatch');
   switch(typeof(Val1))
   {
      case 'object':
         for(var m in Val1)
         {
            if(!Val1.hasOwnProperty(m))
               continue;
            var CurChain=Chain.concat([m]);
            if(!Val2.hasOwnProperty(m))
               return DoThrow('Val2 missing property', CurChain);
            DOC_Val(Val1[m], Val2[m], CurChain);
         }
         return true;
      case 'number':
         if(Number.isNaN(Val1))
            return !Number.isNaN(Val2) ? DoThrow('NaN mismatch') : true;
      case 'string':
      case 'boolean':
         return Val1!==Val2 ? DoThrow('Value mismatch') : true;
      case 'function':
         if(Val1.prototype!==Val2.prototype)
            return DoThrow('Prototype mismatch');
         if(Val1!==Val2)
            return DoThrow('Function mismatch');
         return true;
      default:
         return DoThrow('Val1 is unknown type');
   }
}

63
Updates / DWCF v1.1
« on: September 14, 2016, 10:18:21 pm »
Original update for DWCF v1.1 can be found at https://www.castledragmire.com/Updates/DWCF_v1.1.
Originally posted on: 09/15/16
Regarding: DWCF

DWCF v1.1 updates
     
  • Added functionality:  
         
    • GetVars.VarArray.SQLLookup:    
             
      • Additional string parameters that have %THEVAR% are replaced with the value being checked
      •      
      • If the first item is NULL, it will be removed, and the return will include full rows ("*") of the query result set
      •    
    •  
  •  
  • Updated functionality:  
         
    • json_encode() In RetMsg() and CallByAction() now use JSON_UNESCAPED_UNICODE
    •    
    • GetVars.VarArray.IsOptional now only triggers if it is (boolean)true
    •    
    • Added additional description specification to GetVars.VarArray.SQLLookup which says it uses the additional parameters as values to fill in the SQL Query
    •  
  •  
  • LICENSE: Now applies to 1.x instead of 1.0.x

64
Updates / DSQL v2.0.2
« on: September 14, 2016, 10:13:20 pm »
Original update for DSQL v2.0.2 can be found at https://www.castledragmire.com/Updates/DSQL_v2.0.2.
Originally posted on: 09/15/16
Regarding: DSQL

DSQL v2.0.2
     
  • Added static member $InitialPrintAndDieOnError which DSQL.PrintAndDieOnError inherits on creation
  •  
  • Bug Fixes:  
         
    • mysqli_set_charset is set to utf-8
    •    
    • In FormatSQLError() the date() function used for “Start Time” now uses “24-hour format of an hour with leading zeros” [date(“H”)] instead of “12-hour format of an hour without leading zeros” [date(“g”)]
    •  

65
Posts / Windows Driver Service Loader
« on: August 02, 2016, 08:55:57 pm »
Original post for Windows Driver Service Loader can be found at https://www.castledragmire.com/Posts/Windows_Driver_Service_Loader.
Originally posted on: 08/02/16

Following is some C++ source code for a Windows kernel-driver service loader. It could be used to load other service types too by changing the dwServiceType flag on the CreateService call. I threw this together for another project I am currently working on. It is also used in the following post (posting soon).

It works in the following way:
  • It is a command line utility which takes 3 arguments:
    1. The service name. Hereby referred to as SERVICE_NAME
    2. The service display name. Hereby referred to as DISPLAY_NAME
    3. The driver path (to the .sys file). Hereby referred to as DRIVER_PATH
  • This program (most likely) requires administrative access. There are also some caveats regarding driver code signing requirements that are thoroughly explored elsewhere.
  • It first checks to see if a service already exists with the given SERVICE_NAME. If it does:
    1. If the DISPLAY_NAME matches, the service is kept as is.
    2. If the DISPLAY_NAME does not match, the user is prompted on if they want to delete the current service. If they do not, the program exits.
  • If the service needs to be created (it did not already exist or was deleted), it creates the service with the given SERVICE_NAME, DISPLAY_NAME, and DRIVER_PATH. If the service is not created during this run, the DRIVER_PATH is ignored.
    Note: The DRIVER_PATH must be to a direct local file system file. I have found that network links and symbolic links do not work.
  • The service is started up:
    • If it is already running, the user is prompted on if they want to stop the currently running service. If they say no, the program exits.
  • The program then waits for a final user input on if they want to close the service before exiting the program.
  • If there was an error, the program reports the error, otherwise, it reports “Success”.
  • The program pauses at the end until the user presses any key to exit.
  • The program returns 0 on success, and 1 if an error occurred.

//Compiler flags
#define WIN32_LEAN_AND_MEAN  //Include minimum amount of windows stuff
#ifndef _UNICODE //Everything in this script is unicode
   #define _UNICODE
#endif

#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <memory>

//Smart pointers
typedef std::unique_ptr<WCHAR, void(*)(WCHAR*)> SmartWinAlloc;
typedef std::unique_ptr<SC_HANDLE, void(*)(SC_HANDLE*)> SmartCloseService;
void Delete_SmartWinAlloc(WCHAR *p) { if(p) LocalFree(p); }
void Delete_SmartCloseService(SC_HANDLE *h) { if(h && *h) CloseServiceHandle(*h); }

//Function declarations
WCHAR* InitDriver(int argc, WCHAR *argv[]);
WCHAR* FormatError(WCHAR* Format, ...);
SmartWinAlloc GetLastErrorStr();
BOOLEAN AskQuestion(WCHAR* Question); //Returns if user answered yes

int wmain(int argc, WCHAR *argv[])
{
   //Run the init routine
   WCHAR* Ret=InitDriver(argc, argv);

   //If there is an error, report it, or otherwise, report success
   wprintf(L"%s\n", Ret ? Ret : L"Success");
   wprintf(L"%s\n", L"Press any key to exit");
   _getch();

   //Return if successful
   return (Ret ? 1 : 0);
}

WCHAR* InitDriver(int argc, WCHAR *argv[])
{
   //Confirm arguments
   if(argc<4)
      return FormatError(L"%s", L"3 arguments are required: Service Name, Display Name, Driver Path");
   const WCHAR* Param_ServiceName=argv[1];
   const WCHAR* Param_DisplayName=argv[2];
   const WCHAR* Param_DriverPath =argv[3];

   //Open the service manager
   wprintf(L"%s\n", L"Opening the service manager");
   SC_HANDLE HSCManager=OpenSCManager(nullptr, nullptr, SC_MANAGER_CREATE_SERVICE);
   if(!HSCManager)
      return FormatError(L"%s: %s", L"Error opening service manager", GetLastErrorStr());
   SmartCloseService FreeHSCManager(&HSCManager, Delete_SmartCloseService);

   //Check if the service already exists
   wprintf(L"%s\n", L"Checking previously existing service state");
   BOOL ServiceExists=false;
   {
      //Get the service name
      const DWORD NameBufferSize=255;
      WCHAR NameBuffer[NameBufferSize];
      WCHAR *NamePointer=NameBuffer;
      DWORD NamePointerSize=NameBufferSize;
      std::unique_ptr<WCHAR> Buf(nullptr); //May be swapped with a real pointer later
      for(INT_PTR i=0;i<2;i++)
      {
         //If we found the service, exit the lookup here
         if(GetServiceDisplayName(HSCManager, Param_ServiceName, NamePointer, &NamePointerSize))
         {
            ServiceExists=true;
            break;
         }

         //If the service does not exist, we can exit the lookup here
         if(GetLastError()==ERROR_SERVICE_DOES_NOT_EXIST)
            break;

         //If error is not insufficient buffer size, return the error
         if(GetLastError()!=ERROR_INSUFFICIENT_BUFFER)
            return FormatError(L"%s: %s", L"Could not query service information", GetLastErrorStr());

         //If second pass, error out
         if(i==1)
            return FormatError(L"%s: %s", L"Could not query service information", L"Second buffer pass failed");

         //Create a buffer of appropriate size (and make sure it will later be released)
         NamePointer=new WCHAR[++NamePointerSize];
         std::unique_ptr<WCHAR> Buf2(NamePointer);
         Buf.swap(Buf2);
      }

      //If the service already exists, confirm the service name matches, and if not, ask if user wants to delete the current service
      if(ServiceExists)
      {
         wprintf(L"%s\n", L"The service already exists");
         if(wcsncmp(NamePointer, Param_DisplayName, NamePointerSize+1))
         {
            //If the server names do not match, ask the user what to do
            wprintf(L"%s:\nCurrent: %s\nRequested: %s\n", L"The service names do not match", NamePointer, Param_DisplayName);

            //Make the request
            if(!AskQuestion(L"Would you like to replace the service? (y/n)")) //If user does not wish to replace the service
               return FormatError(L"%s", L"Cannot continue if service names do not match");

            //Delete the service
            wprintf(L"%s\n", L"Deleting the old service");
            ServiceExists=false;
            SC_HANDLE TheService=OpenService(HSCManager, Param_ServiceName, DELETE);
            if(!TheService)
               return FormatError(L"%s: %s", L"Could not open the service to delete it", GetLastErrorStr());
            SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service handle
            if(!DeleteService(TheService))
               return FormatError(L"%s: %s", L"Could not delete the service", GetLastErrorStr());
            wprintf(L"%s\n", L"The service has been deleted");
         }
      }
   }

   //Create the service
   SC_HANDLE TheService;
   if(!ServiceExists)
   {
      //Confirm the driver path exists
      wprintf(L"%s\n", L"Checking the driver file");
      DWORD FileAttrs=GetFileAttributes(Param_DriverPath);
      if(FileAttrs==INVALID_FILE_ATTRIBUTES)
         return FormatError(L"%s: %s", L"Given path is invalid", GetLastErrorStr());
      if(FileAttrs&FILE_ATTRIBUTE_DIRECTORY)
         return FormatError(L"%s: %s", L"Given path is invalid", L"Path is a folder");

      //Create the service
      wprintf(L"%s\n", L"Creating the service");
      TheService=CreateService(
         HSCManager, Param_ServiceName, Param_DisplayName,
         SERVICE_START|SERVICE_STOP,
         SERVICE_KERNEL_DRIVER, SERVICE_DEMAND_START, SERVICE_ERROR_IGNORE,
         Param_DriverPath, nullptr, nullptr, nullptr, nullptr, nullptr);
      if(!TheService)
         return FormatError(L"%s: %s", L"Could not create the service", GetLastErrorStr());

   //Open the service if not creating
   } else {
      TheService=OpenService(HSCManager, Param_ServiceName, SERVICE_START|SERVICE_STOP);
      if(!TheService)
         return FormatError(L"%s: %s", L"Could not open the service", GetLastErrorStr());
   }
   SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service on exit

   //Start the service
   wprintf(L"%s\n", L"Starting the service");
   for(INT_PTR i=0;i<2;i++)
   {
      if(StartService(TheService, 0, nullptr))
         break;

      //If not "service already running" error, or user does not want to stop the current service
      if(i==1 || GetLastError()!=ERROR_SERVICE_ALREADY_RUNNING || !AskQuestion(L"The service is already running. Would you like to stop it? (y/n)"))
         return FormatError(L"%s: %s", L"Could not start the service", GetLastErrorStr());

      //Stop the service
      SERVICE_STATUS ss;
      wprintf(L"%s\n", L"Stopping the current service");
      if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss))
         return FormatError(L"%s: %s", L"Could not stop the current service", GetLastErrorStr());
   }
   wprintf(L"%s\n", L"Started the service");

   //Ask if the user wants to close the service
   if(!AskQuestion(L"Would you like to stop the service before exit? (y/n)"))
      return nullptr;

   //Stop the service
   SERVICE_STATUS ss;
   if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss))
      return FormatError(L"%s: %s", L"Could not stop the service", GetLastErrorStr());
   if(ss.dwCurrentState!=SERVICE_STOP_PENDING && ss.dwCurrentState!=SERVICE_STOPPED)
      return FormatError(L"%s", L"The service does not appear to be closing");
   wprintf(L"%s\n", L"The service has been stopped");

   //Return success
   return nullptr;
}

WCHAR* FormatError(WCHAR* Format, ...)
{
   static WCHAR Err[255];
   va_list VAList;
   va_start(VAList, Format);
   vswprintf(Err, sizeof(Err)/sizeof(Err[0]), Format, VAList);
   return Err;
}

SmartWinAlloc GetLastErrorStr()
{
   LPWSTR MessageBuffer=nullptr;
   FormatMessage(
      FORMAT_MESSAGE_ALLOCATE_BUFFER|FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS|FORMAT_MESSAGE_MAX_WIDTH_MASK,
      nullptr, GetLastError(), MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (LPWSTR)&MessageBuffer, 0, nullptr);
   return SmartWinAlloc(MessageBuffer, Delete_SmartWinAlloc);
}

BOOLEAN AskQuestion(WCHAR* Question)
{
   //Make the request and wait for an input character
   while(1)
   {
      //Ask the question and get the answer
      wprintf(L"%s:", Question);
      fflush(stdout);
      char InputChar=_getch();
      printf("\n");

      //Check for a valid answer
      if(InputChar=='n' || InputChar=='N')
         return FALSE;
      if(InputChar=='y' || InputChar=='Y')
         return TRUE;
   }
}

66
Projects / Re: Plex Playlist Importer
« on: July 26, 2016, 08:12:09 pm »
You mean a separate Plex pass user, correct? One in which you would have had to of shared your libraries with via Libraries->...->Share ? (Or shares visible via plex.tv->launch->settings->users->friends)

67
Updates / Plex Playlist Importer v1.1
« on: July 14, 2016, 08:27:22 pm »
Original update for Plex Playlist Importer v1.1 can be found at https://www.castledragmire.com/Updates/Plex_Playlist_Importer_v1.1.
Originally posted on: 07/14/16
Regarding: Plex Playlist Importer

Plex Playlist Importer v1.1 (source). Current updates spanning 2017:
     
  • Fixed absolute path logic for Linux (partial credit to Matt Spitz)
  •  
  • Bypasses UTF8 BOM
  •  
  • Program now works off of argument flags.
         
    • Added parameters: Playlist encoding, override type, force list
    •    
    • Created special BulletHelpFormatter class for parameters
    •  
  •  
  • Playlist names can now conflict with other item/list names in Plex
  •  
  • Console column width passthrough in the .sh file
  •  
  • Updated READMEs regarding:
         
    • Unicode compliance
    •    
    • The “no such module : FTS4” error
    •    
    • Running the script from a computer external to the server running Plex
    •    
    • The “The program can’t start because MSVCR100.dll is missing” error
    •    
    • All updates
    •  
  •  
  • Added the ability to compile to a windows executable (via setup.py py2exe)
  •  
  • The Playlist Name is now an optional argument which can be entered after the program is ran. This allows directly dragging playlists onto the executable
  •  
  • Added shebang to main script
  •  
  • M3U files now ignore lines that are empty or have only whitespace

68
General Discussion / Re: how to enter the parameters?
« on: July 05, 2016, 04:42:39 pm »
Sure. If you needed the latest executable, it is available in this thread.

69
Projects / Re: Plex Playlist Importer
« on: July 05, 2016, 01:53:02 am »
Yikes. It feels like on a lot of this you are going to way more trouble than should be needed :-\

Quote
I've not found a good way to execute ListImporter from a folder not containing both the .exe and all its .dll files. This would be useful but is not essential. I do have to manage Acronis' insistence on restarting one of its supposedly unneeded services that uses a different sqlite3.dll. The sqlite3.dll used by the latest version of PMS no longer conflicts.
I think I might actually have been providing a bad sqlite3.dll with my previous releases, which could have been the problem. That is fixed in the new archive I had sent you. I don't really see an easier way to do distribute this without the dlls as separate files in the archive. I'm not big on installers.
Quote
The first of two small issues is that ListImporter is intolerant of blank lines in the m3u (e.g., at the end).
I have updated the code to ignore lines that are blank or only have whitespace. It is up on github and I will included it on my next executable install.
Quote
The second small issue is that any error in one track reference causes the entire import to abort with Errorlevel set to 1.  [...]
This is always something I haven't really been sure how to handle. Do you have any suggestions? I guess I could include a batch error mode that would be the default...
Quote
(a) execution message output appears to go to stderr vs. stdout, which can interfere with batch file output piping for logging/analysis purposes
I believe throwing errors to stderr is the proper behavior. Would it help if I included a parameter flag that instead pushed everything through stdout? In bash shell environments, it's as simple as adding "2>&1" after your command. Not sure about windows.
I think that covers everything you mentioned, minus track length. I just looked in the DB, and from a quick glance, I'm not actually sure where the track durations are stored for items on a playlist. I think I skipped that since I wanted to make a minimum number of modifications to the database and knew it would auto correct.

[Edit] P.S. The latest version (have not compiled into exe yet) allows dragging playlists onto the executable.

70
Posts / Painless migration from PHP MySQL to MySQLi
« on: July 02, 2016, 03:52:11 am »

The PHP MySQL extension is being deprecated in favor of the MySQLi extension in PHP 5.5, and removed as of PHP 7.0. MySQLi was first referenced in PHP v5.0.0 beta 4 on 2004-02-12, with the first stable release in PHP 5.0.0 on 2004-07-13[1]. Before that, the PHP MySQL extension was by far the most popular way of interacting with MySQL on PHP, and still was for a very long time after. This website was opened only 2 years after the first stable release!


With the deprecation, problems from some websites I help host have popped up, many of these sites being very, very old. I needed a quick and dirty solution to monkey-patch these websites to use MySQLi without rewriting all their code. The obvious answer is to overwrite the functions with wrappers for MySQLi. The generally known way of doing this is with the Advanced PHP Debugger (APD). However, using this extension has a lot of requirements that are not appropriate for a production web server. Fortunately, another extension I recently learned of offers the renaming functionality; runkit. It was a super simple install for me.

  1. From the command line, run “pecl install runkit”
  2. Add “extension=runkit.so” and “runkit.internal_override=On” to the php.ini

Besides the ability to override these functions with wrappers, I also needed a way to make sure this file was always loaded before all other PHP files. The simple solution for that is adding “auto_prepend_file=/PATH/TO/FILE” to the “.user.ini” in the user’s root web directory.

The code for this script is as follows. It only contains a limited set of the MySQL functions, including some very esoteric ones that the web site used. This is not a foolproof script, but it gets the job done.


//Override the MySQL functions
foreach(Array(
   'connect', 'error', 'fetch_array', 'fetch_row', 'insert_id', 'num_fields', 'num_rows',
   'query', 'select_db', 'field_len', 'field_name', 'field_type', 'list_dbs', 'list_fields',
   'list_tables', 'tablename'
) as $FuncName)
   runkit_function_redefine("mysql_$FuncName", '',
       'return call_user_func_array("mysql_'.$FuncName.'_OVERRIDE", func_get_args());');

//If a connection is not explicitely passed to a mysql_ function, use the last created connection
global $SQLLink; //The remembered SQL Link
function GetConn($PassedConn)
{
   if(isset($PassedConn))
       return $PassedConn;
   global $SQLLink;
   return $SQLLink;
}

//Override functions
function mysql_connect_OVERRIDE($Host, $Username, $Password) {
   global $SQLLink;
   return $SQLLink=mysqli_connect($Host, $Username, $Password);
}
function mysql_error_OVERRIDE($SQLConn=NULL) {
   return mysqli_error(GetConn($SQLConn));
}
function mysql_fetch_array_OVERRIDE($Result, $ResultType=MYSQL_BOTH) {
   return mysqli_fetch_array($Result, $ResultType);
}
function mysql_fetch_row_OVERRIDE($Result) {
   return mysqli_fetch_row($Result);
}
function mysql_insert_id_OVERRIDE($SQLConn=NULL) {
   return mysqli_insert_id(GetConn($SQLConn));
}
function mysql_num_fields_OVERRIDE($Result) {
   return mysqli_num_fields($Result);
}
function mysql_num_rows_OVERRIDE($Result) {
   return mysqli_num_rows($Result);
}
function mysql_query_OVERRIDE($Query, $SQLConn=NULL) {
   return mysqli_query(GetConn($SQLConn), $Query);
}
function mysql_select_db_OVERRIDE($DBName, $SQLConn=NULL) {
   return mysqli_select_db(GetConn($SQLConn), $DBName);
}
function mysql_field_len_OVERRIDE($Result, $Offset) {
   $Fields=$Result->fetch_fields();
   return $Fields[$Offset]->length;
}
function mysql_field_name_OVERRIDE($Result, $Offset) {
   $Fields=$Result->fetch_fields();
   return $Fields[$Offset]->name;
}
function mysql_field_type_OVERRIDE($Result, $Offset) {
   $Fields=$Result->fetch_fields();
   return $Fields[$Offset]->type;
}
function mysql_list_dbs_OVERRIDE($SQLConn=NULL) {
   $Result=mysql_query('SHOW DATABASES', GetConn($SQLConn));
   $Tables=Array();
   while($Row=mysqli_fetch_assoc($Result))
       $Tables[]=$Row['Database'];
   return $Tables;
}
function mysql_list_fields_OVERRIDE($DBName, $TableName, $SQLConn=NULL) {
   $SQLConn=GetConn($SQLConn);
   $CurDB=mysql_fetch_array(mysql_query('SELECT Database()', $SQLConn));
   $CurDB=$CurDB[0];
   mysql_select_db($DBName, $SQLConn);
   $Result=mysql_query("SHOW COLUMNS FROM $TableName", $SQLConn);
   mysql_select_db($CurDB, $SQLConn);
   if(!$Result) {
       print 'Could not run query: '.mysql_error($SQLConn);
       return Array();
   }
   $Fields=Array();
   while($Row=mysqli_fetch_assoc($Result))
       $Fields[]=$Row['Field'];
   return $Fields;
}
function mysql_list_tables_OVERRIDE($DBName, $SQLConn=NULL) {
   $SQLConn=GetConn($SQLConn);
   $CurDB=mysql_fetch_array(mysql_query('SELECT Database()', $SQLConn));
   $CurDB=$CurDB[0];
   mysql_select_db($DBName, $SQLConn);
   $Result=mysql_query("SHOW TABLES", $SQLConn);
   mysql_select_db($CurDB, $SQLConn);
   if(!$Result) {
       print 'Could not run query: '.mysql_error($SQLConn);
       return Array();
   }
   $Tables=Array();
   while($Row=mysql_fetch_row($Result))
       $Tables[]=$Row[0];
   return $Tables;
}
function mysql_tablename_OVERRIDE($Result) {
   $Fields=$Result->fetch_fields();
   return $Fields[0]->table;
}

And here is some test code to confirm functionality:
global $MyConn, $TEST_Table;
$TEST_Server='localhost';
$TEST_UserName='...';
$TEST_Password='...';
$TEST_DB='...';
$TEST_Table='...';
function GetResult() {
   global $MyConn, $TEST_Table;
   return mysql_query('SELECT * FROM '.$TEST_Table.' LIMIT 1', $MyConn);
}
var_dump($MyConn=mysql_connect($TEST_Server, $TEST_UserName, $TEST_Password));
//Set $MyConn to NULL here if you want to test global $SQLLink functionality
var_dump(mysql_select_db($TEST_DB, $MyConn));
var_dump(mysql_query('SELECT * FROM INVALIDTABLE LIMIT 1', $MyConn));
var_dump(mysql_error($MyConn));
var_dump($Result=GetResult());
var_dump(mysql_fetch_array($Result));
$Result=GetResult(); var_dump(mysql_fetch_row($Result));
$Result=GetResult(); var_dump(mysql_num_fields($Result));
var_dump(mysql_num_rows($Result));
var_dump(mysql_field_len($Result, 0));
var_dump(mysql_field_name($Result, 0));
var_dump(mysql_field_type($Result, 0));
var_dump(mysql_tablename($Result));
var_dump(mysql_list_dbs($MyConn));
var_dump(mysql_list_fields($TEST_DB, $TEST_Table, $MyConn));
var_dump(mysql_list_tables($TEST_DB, $MyConn));
mysql_query('CREATE TEMPORARY TABLE mysqltest (i int auto_increment, primary key (i))', $MyConn);
mysql_query('INSERT INTO mysqltest VALUES ()', $MyConn);
mysql_query('INSERT INTO mysqltest VALUES ()', $MyConn);
var_dump(mysql_insert_id($MyConn));
mysql_query('DROP TEMPORARY TABLE mysqltest', $MyConn);

71
Posts / Bullet Help Formatter for Python’s argparse
« on: July 02, 2016, 02:55:20 am »
Original post for Bullet Help Formatter for Python’s argparse can be found at https://www.castledragmire.com/Posts/Bullet_Help_Formatter_for_Python’s_argparse.
Originally posted on: 07/02/16

This is a HelpFormatter for Python’s argparse class which:
  • Takes raw input and wraps long lines to indent against the current line start.
  • When an indented/list line is encountered, which starts with spaces followed by a star "*", wrapped line’s indents will start 2 spaces after the star.
  • Lines attempt to split at words of 10 characters or less (see .MinCharsInSplitWord).
  • If a line needs to split along a word longer than this, a hyphen is inserted at the end of the line.

import argparse
import re
class BulletHelpFormatter(argparse.HelpFormatter):
   def __init__(self, *args, **kwargs):
       super(BulletHelpFormatter, self).__init__(*args, **kwargs)
       self.MinCharsInSplitWord=10

   def _split_lines(self, text, width):
       #Split lines around line breaks and then modify each line
       Lines=[]
       for Line in text.splitlines():
           #Get the number of spaces to put at subsequent lines
           #0 if not a list item, oherwise, 2+list item start
           ListEl=re.match(r'^ *\*', Line)
           NumBeginningSpace=(0 if ListEl==None else ListEl.end()+1)

           #Add extra spaces at the beginning of each line to match the start of the current line, and go to a maxium of $width
           IsFirstPass=True
           SpacesToAdd=''
           NumSpacesToAdd=0
           while(True):
               #Get the word break points before and after where the line would end
               MaxLineLen=max(min(width-NumSpacesToAdd, len(Line)), 1)
               PrevWordBreak=CurWordBreak=0
               for WordBreak in re.finditer(r'(?<=\W).|\W|$', Line):
                   PrevWordBreak=CurWordBreak
                   CurWordBreak=WordBreak.start()
                   if CurWordBreak>=MaxLineLen:
                       if CurWordBreak==MaxLineLen:
                           PrevWordBreak=CurWordBreak
                       break

               #If previous wordbreak is more than MinCharsInSplitWord away from MaxLineLen, then split at the end of the line
               IsSplit=(PrevWordBreak<1 or CurWordBreak-PrevWordBreak>self.MinCharsInSplitWord)
               SplitPos=(MaxLineLen if IsSplit else PrevWordBreak)

               #Append the new line to the list of lines
               Lines.append(SpacesToAdd+Line[0:SplitPos]+('-' if IsSplit else ''))
               Line=Line[SplitPos:]

               #If this is the end, nothing left to do
               if len(Line)==0:
                   break

               #If this is the first pass, update line creation variables
               if IsFirstPass:
                   IsFirstPass=False
                   NumSpacesToAdd=NumBeginningSpace
                   SpacesToAdd=(' ' * NumSpacesToAdd)

       return Lines

72
Projects / Re: Plex Playlist Importer
« on: July 01, 2016, 04:16:31 am »
I have made multiple updates to the software and recompiled it. I have attached it to this post if you want to check it out. I confirmed that it worked fine in command prompt with your m3u file. Update log is on github.

P.S. If possible, please let me know within the next few days if all is well now. Would like to get the confirmed updates and compilation up on my website.

73
General Discussion / Re: how to enter the parameters?
« on: June 29, 2016, 01:41:47 am »
Quote
I guess that french characters is the cause of that error:"List Importer  'Winamp playlist':'utf-8' codec can't decode byte 0xe9 in position  323:invalid continuarion byte
I have added a command line flag that lets you set the encoding of the file. You will want to set it to ISO-8859-1.
Quote
I chose an full English playlist. I have that:"Playlist name has already been taken by a non-playlist"
This has been fixed.
Quote
It seem that my plex playlist name can't be the same as the artist that  is already in my plex library, so I change the playlist name to  "paramorelist":DB Error: no such module fts4
I have added the following to the README file:
Quote
If running this mentions something about “no such module : FTS4”, you may need to replace the sqlite3.dll or sqlite3.so for your Python, which can be found at https://www.sqlite.org/download.html .
For a Python for Windows install, the DLL location will most likely be located at C:\Users\%USERNAME%\AppData\Local\Programs\Python\Python%PYTHON_VERSION%\DLLs.

The new version is available on github. Will be uploading a new windows executable within the next few days.

74
Projects / Re: Plex Playlist Importer
« on: June 25, 2016, 03:24:48 am »
I'm working on updating the software this weekend.

75
General Discussion / Re: how to enter the parameters?
« on: June 25, 2016, 03:24:33 am »
I'm working on updating the software this weekend.

Pages: 1 ... 3 4 [5] 6 7 ... 38