|
-
April 26, 2024, 03:49:46 am
- Welcome, Guest
News:Official site launch very soon, hurrah!
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
Messages - Dakusan
Pages: 1 2 [3] 4 5 ... 36
31
« on: February 17, 2017, 02:47:07 am »
Same disclaimer. Guess I gotta add command line flags for this and update on github. Already have 1 pull request on github. PITA cause I don't release code without thoroughly testing.
Line 137. Change
sys.exit("File not found in DB: "+FilePath)to
continueWhen I update on github it will throw a warning instead.
32
« on: February 16, 2017, 03:56:28 am »
Had a need today to sync file modification timestamps between plex and the actual files. Code is as follows.
#Dump from sqlite (DB should be in C:\Users\USERNAME\AppData\Local\Plex Media Server\Plug-in Support\Databases
sqlite3 com.plexapp.plugins.library.db 'select file, updated_at from media_parts' | \
sort | uniq | perl -pe 's/\\/\//g' | perl -pe 's/^\w:/\./gi' | perl -pe 's/\|/\t/g' | \
grep -viP '^((DRIVES_TO_IGNORE):|\./(DIRECTORIES_TO_IGNORE)|\|$)' > NAME_CHECKS
#Find all files with their modification timestamps
find -L -type f -printf "%p\t%TY-%Tm-%Td %TH:%TM:%.2TS\n" | sort > FTIMES2
#Filter out unwanted folders and file extensions. I did this as a separate command from the above line to allow filtering without having to run a find on the entire drive again
cat FTIMES2 | grep -vP '\.(md5|torrent|sub|idx|nfo|srt|txt|ssa|log|db|jpg|tbn|sfv|png|cbz|rar|cbr|OTHER_EXTENSIONS)\t' | grep -vP '^./(System Volume Information|\$RECYCLE\.BIN|OTHER_FOLDERS)/' > FTIMES
#After comparing the 2 files and extracting any files that need to be updated, run this regular expression on the data to get touch commands to update the timestamps
^(.*)\t\d\d(\d\d)-(\d\d)-(\d\d) (\d\d):(\d\d):(\d\d)$ => touch -m -t $2$3$4$5$6.$7 "$1"
33
« on: February 14, 2017, 10:23:31 am »
In PlexPlaylistImporter.py, below line 177 #Insert the items into the playlistAdd the following line
Cur.execute('DELETE FROM play_queue_generators WHERE playlist_id=?', (PlexPlaylistID,))
I have not tested this code, so make sure to backup your database before attempting it.
34
« on: January 20, 2017, 09:05:24 am »
List joined hard links. Same links are separated by tab with a newline between each set.
find -links +1 | xargs -d"\n" ls -i | perl -e 'foreach $Line (<STDIN>) { @Cols=($Line=~/^\s*(\d+)\s*(.*?)\s*$/); push(@{$Link{$Cols[0]}}, $Cols[1]); } foreach $List (values %Link) { print join("\t", @{$List})."\n"; }'
35
« on: January 06, 2017, 04:15:20 am »
Just something that might be useful to others. I couldn’t find a copy of this anywhere else online. This should contain every point on the map. If anyone finds any more, please let me know! 
36
« on: November 10, 2016, 04:52:15 pm »
Regular expressions have been one of my favorite programming tools since I first discovered them. They are wonderfully robust and things can usually be done with them in many ways. For example, here are multiple ways to match an IPv4 address: - ^\d\d?\d?\.\d\d?\d?\.\d\d?\d?\.\d\d?\d?$
- ^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$
- ^(\d{1,3}\.){3}\d{1,3}$
- ^([0-9]{1,3}\.){3}[0-9]{1,3}$
One of my major annoyances though has always been lists. I have always done them like ^(REGEX,)*REGEX$. For example, I would do a list of IP addresses like this: ^(\d{1,3}\.){3}\d{1,3},)*\d{1,3}\.){3}\d{1,3}$. I recently realized however that a list can much more elegantly be done as follows: ^(REGEX(,|$))+(?<!,)$. I would describe this as working by: - ^: Start of the statement (test string)
- (REGEX(,|$))+: A list of items separated by either a comma or EOS (end of statement). If we keep this regular expression as not-multi-line (the default), then the EOS can only happen at the end of the statement.
- (?<!,): This is a look-behind assertion saying that the last character before the EOS cannot be a comma. If we didn’t have this, the list could look like this, with a comma at the end: “ITEM,ITEM,ITEM,”.
- $: The end of the statement
So the new version of the IP address list would look like this ^((\d{1,3}\.){3}\d{1,3}(,|$))+(?<!,)$ instead of this ^((\d{1,3}\.){3}\d{1,3},)*(\d{1,3}\.){3}\d{1,3}$. Also, since an IP address is just a list of numbers separated by periods, it could also look like this: ^(\d{1,3}(\.|$)){4}(?<!\.)$.
37
« on: October 27, 2016, 02:46:15 am »
I wanted to write about a really weird problem I recently had while debugging in C++ (technically, it’s all C). Unfortunately, I was doing this in kernel debugging mode, which made life a bit harder, but it would have happened the same in userland. I had an .hpp file (we’ll call it process_internal.hpp) that was originally an internal file just to be included from a .cpp file (we’ll call it process.cpp), so it contained global variables as symbols. I ended up needing to include this process_internal.hpp file elsewhere (for testing, we’ll call it test.cpp). Because of this, the same symbol was included in multiple files, so the separate .o builds were not properly interacting. I ended up using “#ifdef”s to only include the parts I needed in the test.cpp file, and doing “extern” defines of the global variables for it. It looked something like the following:
enum { FT_Inbound, FT_Outbound };
typedef struct FilteringLayer {
int FilterTypeNum, OriginalID;
const char *Name;
} FilteringLayer;
const int FT_NumTypes=2;
#ifdef _PROCESS_INTERNAL
FilteringLayer FilterTypes[FT_NumTypes]={
{FT_Inbound, 5, "Inbound"),
{FT_Outbound, 8, "Outbound"),
};
#else
extern "C" FilteringLayer *FilterTypes;
#endif
So I was accessing this variable in test.cpp and getting a really weird problem. The code looked something like this:
struct foo { int a, b; };
foo Stuff[]={...};
void FunctionBar()
{
for(int i=0;i<FT_NumTypes;i++)
Stuff[FilterTypes[i].OriginalID].b=1;
}
This was causing an access exception, which blue screened my debug VM. I tried running the exact same statements in the visual studio debugger, and things were working just as they were supposed to! So I decided to go to the assembly level. It looked something like this: (I included descriptions) | L# | Code | | | | 1 | mov qword ptr [rsp+58h],0 | | 2 | jmp MODULENAME!FunctionBar+0xef | | 3 | mov rax,qword ptr [rsp+58h] | | 4 | inc rax | | 5 | mov qword ptr [rsp+58h],rax | | 6 | cmp qword ptr [rsp+58h],02h | | 7 | jae MODULENAME!FunctionBar+0x11e | | | | 8 | imul rax,qword ptr [rsp+58h],10h | | 9 | mov rcx,[MODULENAME!FilterTypes ] | | 10 | movzx eax,word ptr [rcx+rax+4] | | 11 | imul rax,rax,30h | | 12 | lea rcx,[MODULENAME!Stuff ] | | 13 | mov dword ptr [rcx+rax+04h],1 | | 14 | jmp MODULENAME!FunctionBar+0xe2 | | 15 |
I noticed that line #9 was putting 0x0000000C`00000000 into RCX instead of &FilterTypes. I knew the instruction should have been an “lea” instead of a “mov” to fix this. My first thought was compiler bug, but as many programming mantras say, that is very very rarely the case. If you want to guess now what the problem is, now is the time. I’ve given you all the information (and more) to make the guess. The answer: extern "C" FilteringLayer *FilterTypes; should have been extern "C" FilteringLayer FilterTypes[];. Oops! The debugger was getting it right because it had the extra information of the real definition of the FilterTypes variable.
38
« on: October 03, 2016, 09:43:08 am »
There are 3 different methods for updating multiple rows at once in MySQL with different values: - INSERT: INSERT with ON DUPLICATE KEY UPDATE
INSERT INTO FooBar (ID, foo)
VALUES (1, 5), (2,  , (3, 2)
ON DUPLICATE KEY UPDATE foo=VALUES(foo);
- TRANSACTION: Where you do an update for each record within a transaction (InnoDB or other DBs with transactions)
START TRANSACTION;
UPDATE FooBar SET foo=5 WHERE ID=1;
UPDATE FooBar SET foo=8 WHERE ID=2;
UPDATE FooBar SET foo=2 WHERE ID=3;
COMMIT;
- CASE: In which you a case/when for each different record within an UPDATE
UPDATE FooBar SET foo=CASE ID
WHEN 1 THEN 5
WHEN 2 THEN 8
WHEN 3 THEN 2
END
WHERE ID IN (1,2,3);
I feel knowing the speeds of the 3 different methods is important. All of the following numbers apply to InnoDB. I just tested this, and the INSERT method was 6.7x faster for me than the TRANSACTION method. I tried on a set of both 3,000 and 30,000 rows and got the same results. The TRANSACTION method still has to run each individually query, which takes time, though it batches the results in memory, or something, while executing. The TRANSACTION method is also pretty expensive in both replication and query logs. Even worse, the CASE method was 41.1x slower than the INSERT method w/ 30,000 records (6.1x slower than TRANSACTION). And 75x slower in MyISAM. INSERT and CASE methods broke even at ~1,000 records. Even at 100 records, the CASE method is BARELY faster. So in general, I feel the INSERT method is both best and easiest to use. The queries are smaller and easier to read and only take up 1 query of action. This applies to both InnoDB and MyISAM. Bonus stuff:Using the INSERT method, there can be a problem in which NON-NULL fields with no default (in other words, required fields) are not being updated. You will get an error like “Field 'fieldname' doesn't have a default value”. The solution is to temporarily turn off STRICT_TRANS_TABLES and STRICT_ALL_TABLES in the SQL mode: SET SESSION sql_mode=REPLACE(REPLACE(@@SESSION.sql_mode,"STRICT_TRANS_TABLES",""),"STRICT_ALL_TABLES",""). Make sure to save the sql_mode first if you plan on reverting it. As for other comments I’ve seen that say the auto_increment goes up using the INSERT method, I tested that too and it seems to not be the case. Code to run the tests is as follows: (It also outputs .SQL files to remove PHP interpreter overhead)
<?
//Variables
$NumRows=30000;
//These 2 functions need to be filled in
function InitSQL()
{
}
function RunSQLQuery($Q)
{
}
//Run the 3 tests
InitSQL();
for($i=0;$i<3;$i++)
RunTest($i, $NumRows);
function RunTest($TestNum, $NumRows)
{
$TheQueries=Array();
$DoQuery=function($Query) use (&$TheQueries)
{
RunSQLQuery($Query);
$TheQueries[]=$Query;
};
$TableName='Test';
$DoQuery('DROP TABLE IF EXISTS '.$TableName);
$DoQuery('CREATE TABLE '.$TableName.' (i1 int NOT NULL AUTO_INCREMENT, i2 int NOT NULL, primary key (i1)) ENGINE=InnoDB');
$DoQuery('INSERT INTO '.$TableName.' (i2) VALUES ('.implode('), (', range(2, $NumRows+1)).')');
if($TestNum==0)
{
$TestName='Transaction';
$Start=microtime(true);
$DoQuery('START TRANSACTION');
for($i=1;$i<=$NumRows;$i++)
$DoQuery('UPDATE '.$TableName.' SET i2='.(($i+5)*1000).' WHERE i1='.$i);
$DoQuery('COMMIT');
}
if($TestNum==1)
{
$TestName='Insert';
$Query=Array();
for($i=1;$i<=$NumRows;$i++)
$Query[]=sprintf("(%d,%d)", $i, (($i+5)*1000));
$Start=microtime(true);
$DoQuery('INSERT INTO '.$TableName.' VALUES '.implode(', ', $Query).' ON DUPLICATE KEY UPDATE i2=VALUES(i2)');
}
if($TestNum==2)
{
$TestName='Case';
$Query=Array();
for($i=1;$i<=$NumRows;$i++)
$Query[]=sprintf('WHEN %d THEN %d', $i, (($i+5)*1000));
$Start=microtime(true);
$DoQuery("UPDATE $TableName SET i2=CASE i1\n".implode("\n", $Query)."\nEND\nWHERE i1 IN (".implode(',', range(1, $NumRows)).')');
}
print "$TestName: ".(microtime(true)-$Start)."
\n";
file_put_contents("./$TestName.sql", implode(";\n", $TheQueries).';');
}
39
« on: September 27, 2016, 02:08:36 pm »
- DSQL v2.0.2.2
- Added ability via DSQL->$StrictMode variable to update MySQL strict mode
- Added DSQL->RawQuery() function which takes just a query, and does no modification on it. I found this required due to extremely slow times on building very large queries.
- Added DSQL->EscapeString() function which escapes a string for use in a mysql query. It has an optional parameter that adds single quotes around the result.
- Added DSQL::PrepareInsertList() function which takes an array of names, and turns it into the following string: ['a', 'b', 'c'] => “(`a`, `b`, `c`) VALUES (?, ?, ?)”
- DWCF v1.1.1
- Added previous variable lookup to GetVars.VarArray.SQLLookup via %THEVAR-$VARNAME%
40
« on: September 27, 2016, 06:04:18 am »
I recently had a Linux client that was, for whatever odd reason, making infinite recursive HTTP calls to a single script, which was making the server process count skyrocket. I decided to use the same module as I did in my Painless migration from PHP MySQL to MySQLi post, which is to say, overriding base functions for fun and profit using the PHP runkit extension. I did this so I could gather, for debugging, logs of when and where the calls that were causing this to occur. The below code overrides all functions listed on the line that says “List of functions to intercept” [Line 9]. It works by first renaming these built in functions to “OVERRIDE_$FuncName” [Line 12], and replacing them with a call to “GlobalRunFunc()” [Line 13], which receives the original function name and argument list. The GlobalRunFunc(): - Checks to see if it is interested in logging the call
- In the case of this example, it will log the call if [Line 20]:
- Line 21: curl_setopt is called with the CURLOPT_URL parameter (enum=10002)
- Line 22: curl_init is called with a first parameter, which would be a URL
- Line 23: file_get_contents or fopen is called and is not an absolute path
(Wordpress calls everything absolutely. Normally I would have only checked for http[s] calls).
- If it does want to log the call, it stores it in a global array (which holds all the calls we will want to log).
The logged data includes [Line 25]:- The function name
- The function parameters
- 2 functions back of backtrace (which can often get quite large when stored in the log file)
- It then calls the original function, with parameters intact, and passes through the return [Line 27].
The “GlobalShutdown()” [Line 30] is then called when the script is closing [Line 38] and saves all the logs, if any exist, to “$GlobalLogDir/$DATETIME.srl”. I have it using “serialize()” to encode the log data [Line 25], as opposed to “json_encode()” or “print_r()” calls, as the latter were getting too large for the logs. You may want to have it use one of these other encoding functions for easier log perusal, if running out of space is not a concern.
<?
//The log data to save is stored here
global $GlobalLogArr, $GlobalLogDir;
$GlobalLogArr=Array();
$GlobalLogDir='./LOG_DIRECTORY_NAME';
//Override the functions here to instead have them call to GlobalRunFunc, which will in turn call the original functions
foreach(Array(
'fopen', 'file_get_contents', 'curl_init', 'curl_setopt', //List of functions to intercept
) as $FuncName)
{
runkit_function_rename($FuncName, "OVERRIDE_$FuncName");
runkit_function_add($FuncName, '', "return GlobalRunFunc('$FuncName', func_get_args());");
}
//This optionally
function GlobalRunFunc($FuncName, $Args)
{
global $GlobalLogArr;
if(
($FuncName=='curl_setopt' && $Args[1]==10002) || //CURLOPT enumeration can be found at https://curl.haxx.se/mail/archive-2004-07/0100.html
($FuncName=='curl_init' && isset($Args[0])) ||
(($FuncName=='file_get_contents' || $FuncName=='fopen') && $Args[0][0]!='/')
)
$GlobalLogArr[]=serialize(Array('FuncName'=>$FuncName, 'Args'=>$Args, 'Trace'=>array_slice(debug_backtrace(), 1, 2)));
return call_user_func_array("OVERRIDE_$FuncName", $Args);
}
function GlobalShutdown()
{
global $GlobalLogArr, $GlobalLogDir;
$Time=microtime(true);
if(count($GlobalLogArr))
file_put_contents($GlobalLogDir.date('Y-m-d_H:i:s.'.substr($Time-floor($Time), 2, 3), floor($Time)).'.srl', implode("\n", $GlobalLogArr));
}
register_shutdown_function('GlobalShutdown');
?>
41
« on: September 26, 2016, 12:37:37 am »
I wrote the code at the end of this post to test the different forms of string concatenation and they really are all almost exactly equal in both memory and time footprints. The two primary methods I used are concatenating strings onto each other, and filling an array with strings and then imploding them. I did 500 string additions with a 1MB string in PHP 5.6 (so the result is a 500MB string). At every iteration of the test, all memory and time footprints were very very close (at ~$IterationNumber*1MB). The runtime of both tests was 50.398 seconds and 50.843 seconds consecutively which are most likely within acceptable margins of error. Garbage collection of strings that are no longer referenced seems to be pretty immediate, even without ever leaving the scope. Since the strings are mutable, no extra memory is really required after the fact. HOWEVER, The following tests showed that there is a different in peak memory usage WHILE the strings are being concatenated.
$OneMB=str_repeat('x', 1024*1024);
$Final=$OneMB.$OneMB.$OneMB.$OneMB.$OneMB;
print memory_get_peak_usage();
Result=10,806,800 bytes (~10MB w/o the initial PHP memory footprint)
$OneMB=str_repeat('x', 1024*1024);
$Final=implode('', Array($OneMB, $OneMB, $OneMB, $OneMB, $OneMB));
print memory_get_peak_usage();
Result=6,613,320 bytes (~6MB w/o the initial PHP memory footprint) So there is in fact a difference that could be significant in very very large string concatenations memory-wise (I have run into such examples when creating very large data sets or SQL queries). But even this fact is disputable depending upon the data. For example, concatenating 1 character onto a string to get 50 million bytes (so 50 million iterations) took a maximum amount of 50,322,512 bytes (~48MB) in 5.97 seconds. While doing the array method ended up using 7,337,107,176 bytes (~6.8GB) to create the array in 12.1 seconds, and then took an extra 4.32 seconds to combine the strings from the array. Anywho... the below is the benchmark code I mentioned at the beginning which shows the methods are pretty much equal. It outputs a pretty HTML table.
<?
//Please note, for the recursion test to go beyond 256, xdebug.max_nesting_level needs to be raised.
//You also may need to update your memory_limit depending on the number of iterations
//Output the start memory
print 'Start: '.memory_get_usage()."B
Below test results are in MB
";
//Our 1MB string
global $OneMB, $NumIterations;
$OneMB=str_repeat('x', 1024*1024);
$NumIterations=500;
//Run the tests
$ConcatTest=RunTest('ConcatTest');
$ImplodeTest=RunTest('ImplodeTest');
$RecurseTest=RunTest('RecurseTest');
//Output the results in a table
OutputResults(
Array('ConcatTest', 'ImplodeTest', 'RecurseTest'),
Array($ConcatTest, $ImplodeTest, $RecurseTest)
);
//Start a test run by initializing the array that will hold the results and manipulating those results after the test is complete
function RunTest($TestName)
{
$CurrentTestNums=Array();
$TestStartMem=memory_get_usage();
$StartTime=microtime(true);
RunTestReal($TestName, $CurrentTestNums, $StrLen);
$CurrentTestNums[]=memory_get_usage();
//Subtract $TestStartMem from all other numbers
foreach($CurrentTestNums as &$Num)
$Num-=$TestStartMem;
unset($Num);
$CurrentTestNums[]=$StrLen;
$CurrentTestNums[]=microtime(true)-$StartTime;
return $CurrentTestNums;
}
//Initialize the test and store the memory allocated at the end of the test, with the result
function RunTestReal($TestName, &$CurrentTestNums, &$StrLen)
{
$R=$TestName($CurrentTestNums);
$CurrentTestNums[]=memory_get_usage();
$StrLen=strlen($R);
}
//Concatenate 1MB string over and over onto a single string
function ConcatTest(&$CurrentTestNums)
{
global $OneMB, $NumIterations;
$Result='';
for($i=0;$i<$NumIterations;$i++)
{
$Result.=$OneMB;
$CurrentTestNums[]=memory_get_usage();
}
return $Result;
}
//Create an array of 1MB strings and then join w/ an implode
function ImplodeTest(&$CurrentTestNums)
{
global $OneMB, $NumIterations;
$Result=Array();
for($i=0;$i<$NumIterations;$i++)
{
$Result[]=$OneMB;
$CurrentTestNums[]=memory_get_usage();
}
return implode('', $Result);
}
//Recursively add strings onto each other
function RecurseTest(&$CurrentTestNums, $TestNum=0)
{
Global $OneMB, $NumIterations;
if($TestNum==$NumIterations)
return '';
$NewStr=RecurseTest($CurrentTestNums, $TestNum+1).$OneMB;
$CurrentTestNums[]=memory_get_usage();
return $NewStr;
}
//Output the results in a table
function OutputResults($TestNames, $TestResults)
{
global $NumIterations;
print '<table border=1 cellspacing=0 cellpadding=2><tr><th>Test Name</th><th>'.implode('</th><th>', $TestNames).'</th></tr>';
$FinalNames=Array('Final Result', 'Clean');
for($i=0;$i<$NumIterations+2;$i++)
{
$TestName=($i<$NumIterations ? $i : $FinalNames[$i-$NumIterations]);
print "<tr><th>$TestName</th>";
foreach($TestResults as $TR)
printf('<td>%07.4f</td>', $TR[$i]/1024/1024);
print '</tr>';
}
//Other result numbers
print '<tr><th>Final String Size</th>';
foreach($TestResults as $TR)
printf('<td>%d</td>', $TR[$NumIterations+2]);
print '</tr><tr><th>Runtime</th>';
foreach($TestResults as $TR)
printf('<td>%s</td>', $TR[$NumIterations+3]);
print '</tr></table>';
}
?>
42
« on: September 24, 2016, 02:40:27 am »
function DeepObjectCompare(O1, O2)
{
try {
DOC_Val(O1, O2, ['O1->O2', O1, O2]);
return DOC_Val(O2, O1, ['O2->O1', O1, O2]);
} catch(e) {
console.log(e.Chain);
throw(e);
}
}
function DOC_Error(Reason, Chain, Val1, Val2)
{
this.Reason=Reason;
this.Chain=Chain;
this.Val1=Val1;
this.Val2=Val2;
}
function DOC_Val(Val1, Val2, Chain)
{
function DoThrow(Reason, NewChain) { throw(new DOC_Error(Reason, NewChain!==undefined ? NewChain : Chain, Val1, Val2)); }
if(typeof(Val1)!==typeof(Val2))
return DoThrow('Type Mismatch');
if(Val1===null || Val1===undefined)
return Val1!==Val2 ? DoThrow('Null/undefined mismatch') : true;
if(Val1.constructor!==Val2.constructor)
return DoThrow('Constructor mismatch');
switch(typeof(Val1))
{
case 'object':
for(var m in Val1)
{
if(!Val1.hasOwnProperty(m))
continue;
var CurChain=Chain.concat([m]);
if(!Val2.hasOwnProperty(m))
return DoThrow('Val2 missing property', CurChain);
DOC_Val(Val1[m], Val2[m], CurChain);
}
return true;
case 'number':
if(Number.isNaN(Val1))
return !Number.isNaN(Val2) ? DoThrow('NaN mismatch') : true;
case 'string':
case 'boolean':
return Val1!==Val2 ? DoThrow('Value mismatch') : true;
case 'function':
if(Val1.prototype!==Val2.prototype)
return DoThrow('Prototype mismatch');
if(Val1!==Val2)
return DoThrow('Function mismatch');
return true;
default:
return DoThrow('Val1 is unknown type');
}
}
43
« on: September 14, 2016, 10:18:21 pm »
DWCF v1.1 updates - Added functionality:
- GetVars.VarArray.SQLLookup:
- Additional string parameters that have %THEVAR% are replaced with the value being checked
- If the first item is NULL, it will be removed, and the return will include full rows ("*") of the query result set
- Updated functionality:
- json_encode() In RetMsg() and CallByAction() now use JSON_UNESCAPED_UNICODE
- GetVars.VarArray.IsOptional now only triggers if it is (boolean)true
- Added additional description specification to GetVars.VarArray.SQLLookup which says it uses the additional parameters as values to fill in the SQL Query
- LICENSE: Now applies to 1.x instead of 1.0.x
44
« on: September 14, 2016, 10:13:20 pm »
DSQL v2.0.2 - Added static member $InitialPrintAndDieOnError which DSQL.PrintAndDieOnError inherits on creation
- Bug Fixes:
- mysqli_set_charset is set to utf-8
- In FormatSQLError() the date() function used for “Start Time” now uses “24-hour format of an hour with leading zeros” [date(“H”)] instead of “12-hour format of an hour without leading zeros” [date(“g”)]
45
« on: August 02, 2016, 08:55:57 pm »
Following is some C++ source code for a Windows kernel-driver service loader. It could be used to load other service types too by changing the dwServiceType flag on the CreateService call. I threw this together for another project I am currently working on. It is also used in the following post (posting soon). It works in the following way: - It is a command line utility which takes 3 arguments:
- The service name. Hereby referred to as SERVICE_NAME
- The service display name. Hereby referred to as DISPLAY_NAME
- The driver path (to the .sys file). Hereby referred to as DRIVER_PATH
- This program (most likely) requires administrative access. There are also some caveats regarding driver code signing requirements that are thoroughly explored elsewhere.
- It first checks to see if a service already exists with the given SERVICE_NAME. If it does:
- If the DISPLAY_NAME matches, the service is kept as is.
- If the DISPLAY_NAME does not match, the user is prompted on if they want to delete the current service. If they do not, the program exits.
- If the service needs to be created (it did not already exist or was deleted), it creates the service with the given SERVICE_NAME, DISPLAY_NAME, and DRIVER_PATH. If the service is not created during this run, the DRIVER_PATH is ignored.
Note: The DRIVER_PATH must be to a direct local file system file. I have found that network links and symbolic links do not work. - The service is started up:
- If it is already running, the user is prompted on if they want to stop the currently running service. If they say no, the program exits.
- The program then waits for a final user input on if they want to close the service before exiting the program.
- If there was an error, the program reports the error, otherwise, it reports “Success”.
- The program pauses at the end until the user presses any key to exit.
- The program returns 0 on success, and 1 if an error occurred.
//Compiler flags
#define WIN32_LEAN_AND_MEAN //Include minimum amount of windows stuff
#ifndef _UNICODE //Everything in this script is unicode
#define _UNICODE
#endif
#include <windows.h>
#include <stdio.h>
#include <conio.h>
#include <memory>
//Smart pointers
typedef std::unique_ptr<WCHAR, void(*)(WCHAR*)> SmartWinAlloc;
typedef std::unique_ptr<SC_HANDLE, void(*)(SC_HANDLE*)> SmartCloseService;
void Delete_SmartWinAlloc(WCHAR *p) { if(p) LocalFree(p); }
void Delete_SmartCloseService(SC_HANDLE *h) { if(h && *h) CloseServiceHandle(*h); }
//Function declarations
WCHAR* InitDriver(int argc, WCHAR *argv[]);
WCHAR* FormatError(WCHAR* Format, ...);
SmartWinAlloc GetLastErrorStr();
BOOLEAN AskQuestion(WCHAR* Question); //Returns if user answered yes
int wmain(int argc, WCHAR *argv[])
{
//Run the init routine
WCHAR* Ret=InitDriver(argc, argv);
//If there is an error, report it, or otherwise, report success
wprintf(L"%s\n", Ret ? Ret : L"Success");
wprintf(L"%s\n", L"Press any key to exit");
_getch();
//Return if successful
return (Ret ? 1 : 0);
}
WCHAR* InitDriver(int argc, WCHAR *argv[])
{
//Confirm arguments
if(argc<4)
return FormatError(L"%s", L"3 arguments are required: Service Name, Display Name, Driver Path");
const WCHAR* Param_ServiceName=argv[1];
const WCHAR* Param_DisplayName=argv[2];
const WCHAR* Param_DriverPath =argv[3];
//Open the service manager
wprintf(L"%s\n", L"Opening the service manager");
SC_HANDLE HSCManager=OpenSCManager(nullptr, nullptr, SC_MANAGER_CREATE_SERVICE);
if(!HSCManager)
return FormatError(L"%s: %s", L"Error opening service manager", GetLastErrorStr());
SmartCloseService FreeHSCManager(&HSCManager, Delete_SmartCloseService);
//Check if the service already exists
wprintf(L"%s\n", L"Checking previously existing service state");
BOOL ServiceExists=false;
{
//Get the service name
const DWORD NameBufferSize=255;
WCHAR NameBuffer[NameBufferSize];
WCHAR *NamePointer=NameBuffer;
DWORD NamePointerSize=NameBufferSize;
std::unique_ptr<WCHAR> Buf(nullptr); //May be swapped with a real pointer later
for(INT_PTR i=0;i<2;i++)
{
//If we found the service, exit the lookup here
if(GetServiceDisplayName(HSCManager, Param_ServiceName, NamePointer, &NamePointerSize))
{
ServiceExists=true;
break;
}
//If the service does not exist, we can exit the lookup here
if(GetLastError()==ERROR_SERVICE_DOES_NOT_EXIST)
break;
//If error is not insufficient buffer size, return the error
if(GetLastError()!=ERROR_INSUFFICIENT_BUFFER)
return FormatError(L"%s: %s", L"Could not query service information", GetLastErrorStr());
//If second pass, error out
if(i==1)
return FormatError(L"%s: %s", L"Could not query service information", L"Second buffer pass failed");
//Create a buffer of appropriate size (and make sure it will later be released)
NamePointer=new WCHAR[++NamePointerSize];
std::unique_ptr<WCHAR> Buf2(NamePointer);
Buf.swap(Buf2);
}
//If the service already exists, confirm the service name matches, and if not, ask if user wants to delete the current service
if(ServiceExists)
{
wprintf(L"%s\n", L"The service already exists");
if(wcsncmp(NamePointer, Param_DisplayName, NamePointerSize+1))
{
//If the server names do not match, ask the user what to do
wprintf(L"%s:\nCurrent: %s\nRequested: %s\n", L"The service names do not match", NamePointer, Param_DisplayName);
//Make the request
if(!AskQuestion(L"Would you like to replace the service? (y/n)")) //If user does not wish to replace the service
return FormatError(L"%s", L"Cannot continue if service names do not match");
//Delete the service
wprintf(L"%s\n", L"Deleting the old service");
ServiceExists=false;
SC_HANDLE TheService=OpenService(HSCManager, Param_ServiceName, DELETE);
if(!TheService)
return FormatError(L"%s: %s", L"Could not open the service to delete it", GetLastErrorStr());
SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service handle
if(!DeleteService(TheService))
return FormatError(L"%s: %s", L"Could not delete the service", GetLastErrorStr());
wprintf(L"%s\n", L"The service has been deleted");
}
}
}
//Create the service
SC_HANDLE TheService;
if(!ServiceExists)
{
//Confirm the driver path exists
wprintf(L"%s\n", L"Checking the driver file");
DWORD FileAttrs=GetFileAttributes(Param_DriverPath);
if(FileAttrs==INVALID_FILE_ATTRIBUTES)
return FormatError(L"%s: %s", L"Given path is invalid", GetLastErrorStr());
if(FileAttrs&FILE_ATTRIBUTE_DIRECTORY)
return FormatError(L"%s: %s", L"Given path is invalid", L"Path is a folder");
//Create the service
wprintf(L"%s\n", L"Creating the service");
TheService=CreateService(
HSCManager, Param_ServiceName, Param_DisplayName,
SERVICE_START|SERVICE_STOP,
SERVICE_KERNEL_DRIVER, SERVICE_DEMAND_START, SERVICE_ERROR_IGNORE,
Param_DriverPath, nullptr, nullptr, nullptr, nullptr, nullptr);
if(!TheService)
return FormatError(L"%s: %s", L"Could not create the service", GetLastErrorStr());
//Open the service if not creating
} else {
TheService=OpenService(HSCManager, Param_ServiceName, SERVICE_START|SERVICE_STOP);
if(!TheService)
return FormatError(L"%s: %s", L"Could not open the service", GetLastErrorStr());
}
SmartCloseService CloseTheService(&TheService, Delete_SmartCloseService); //Close the service on exit
//Start the service
wprintf(L"%s\n", L"Starting the service");
for(INT_PTR i=0;i<2;i++)
{
if(StartService(TheService, 0, nullptr))
break;
//If not "service already running" error, or user does not want to stop the current service
if(i==1 || GetLastError()!=ERROR_SERVICE_ALREADY_RUNNING || !AskQuestion(L"The service is already running. Would you like to stop it? (y/n)"))
return FormatError(L"%s: %s", L"Could not start the service", GetLastErrorStr());
//Stop the service
SERVICE_STATUS ss;
wprintf(L"%s\n", L"Stopping the current service");
if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss))
return FormatError(L"%s: %s", L"Could not stop the current service", GetLastErrorStr());
}
wprintf(L"%s\n", L"Started the service");
//Ask if the user wants to close the service
if(!AskQuestion(L"Would you like to stop the service before exit? (y/n)"))
return nullptr;
//Stop the service
SERVICE_STATUS ss;
if(!ControlService(TheService, SERVICE_CONTROL_STOP, &ss))
return FormatError(L"%s: %s", L"Could not stop the service", GetLastErrorStr());
if(ss.dwCurrentState!=SERVICE_STOP_PENDING && ss.dwCurrentState!=SERVICE_STOPPED)
return FormatError(L"%s", L"The service does not appear to be closing");
wprintf(L"%s\n", L"The service has been stopped");
//Return success
return nullptr;
}
WCHAR* FormatError(WCHAR* Format, ...)
{
static WCHAR Err[255];
va_list VAList;
va_start(VAList, Format);
vswprintf(Err, sizeof(Err)/sizeof(Err[0]), Format, VAList);
return Err;
}
SmartWinAlloc GetLastErrorStr()
{
LPWSTR MessageBuffer=nullptr;
FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER|FORMAT_MESSAGE_FROM_SYSTEM|FORMAT_MESSAGE_IGNORE_INSERTS|FORMAT_MESSAGE_MAX_WIDTH_MASK,
nullptr, GetLastError(), MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), (LPWSTR)&MessageBuffer, 0, nullptr);
return SmartWinAlloc(MessageBuffer, Delete_SmartWinAlloc);
}
BOOLEAN AskQuestion(WCHAR* Question)
{
//Make the request and wait for an input character
while(1)
{
//Ask the question and get the answer
wprintf(L"%s:", Question);
fflush(stdout);
char InputChar=_getch();
printf("\n");
//Check for a valid answer
if(InputChar=='n' || InputChar=='N')
return FALSE;
if(InputChar=='y' || InputChar=='Y')
return TRUE;
}
}
Pages: 1 2 [3] 4 5 ... 36
|
|
