Home Page
  • May 13, 2024, 01:24:33 pm *
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

Official site launch very soon, hurrah!


Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Dakusan

Pages: 1 ... 4 5 [6] 7 8 ... 36
76
Posts / Re: Useful Bash commands and scripts
« on: January 30, 2016, 01:16:39 am »
Dumping a plex database. This includes:
  • Episode path
  • Episode name
  • Episode number
  • Hints (string containing season and episode numbers, and some other info)
Code: [Select]
AppDirectory='/cygdrive/c/Users/Administrator/AppData/Local/Plex Media Server/';
sqlite3 "$AppDirectory/Plug-in Support/Databases/com.plexapp.plugins.library.db" 'SELECT file, title, MDI."index", hints FROM media_parts AS MP INNER JOIN media_items AS MI ON MI.id=MP.media_item_id INNER JOIN metadata_items AS MDI ON MDI.id=MI.metadata_item_id;'[/code

77
Posts / Getting HTML from Simple Machine Forum (SMF) Posts
« on: January 30, 2016, 12:02:56 am »

When I first created my website 10 years ago, from scratch, I did not want to deal with writing a comment system with HTML markups. And in those days, there weren’t plugins for everything like there is today. My solution was setting up a forum which would contain a topic for every Project, Update, and Post, and have my pages mirror the linked topic’s posts.

I had just put in a quick hack at the time in which the pulled SMF message’s body had links converted from bbcode (there might have been 1 other bbcode I also hooked). I had done this with regular expressions, which was a nasty hack.

So anywho, I finally got around to writing a script that converts SMF messages’ bbcode to HTML and caches it. You can download it here, or see the code below. The script is optimized so that it only ever needs to load SMF code when a post has not yet been cached. Caching happens during the initial loading of an SMF post within the script’s main function, and is discarded if the post is changed.

The script requires that you run the query on line #3 of itself in your SMF database. Directly after that are 3 variables you need to set. The script assumes you are already logged in to the appropriate user. To use it, call “GFTP\GetForumTopicPosts($ForumTopicID)”. I have the functions split up so you can do individual posts too if needed (requires a little extra code).


<?
//This SQL command must be ran before using the script
//ALTER TABLE smf_messages ADD body_html text, ADD body_md5 char(32) DEFAULT NULL;

namespace GFTP;

//Forum database variables
global $ForumInfo;
$ForumInfo=Array(
   'DBName'=>'YourDatabase_smf',
   'Location'=>'/home/YourUser/www',
   'MessageTableName'=>'smf2_messages',
);

function GetForumTopicPosts($ForumTopicID)
{
   //Change to the forum database
   global $ForumInfo;
   $CurDB=mysql_fetch_row(mysql_query('SELECT database()'))[0];
   if($CurDB!=$ForumInfo['DBName'])
       mysql_select_db($ForumInfo['DBName']);
   $OldEncoding=SetEncoding(true);

   //Get the posts
   $PostsInfos=Array();
   $PostsQuery=mysql_query('SELECT '.implode(', ', PostFields())." FROM $ForumInfo[MessageTableName] WHERE id_topic='".intval($ForumTopicID).
       "' AND approved=1 ORDER BY id_msg ASC LIMIT 1, 9999999");
   if($PostsQuery) //If query failed, do not process
       while(($PostInfo=mysql_fetch_assoc($PostsQuery)) && ($PostsInfos[]=$PostInfo))
           if(md5($PostInfo['body'])!=$PostInfo['body_md5']) //If the body md5s do not match, get new value, otherwise, use cached value
               ProcessPost($PostsInfos[count($PostsInfos)-1]); //Process the lastest post as a reference

   //Restore from the forum database
   if($CurDB!=$ForumInfo['DBName'])
       mysql_select_db($CurDB);
   SetEncoding(false, $OldEncoding);

   //Return the posts
   return $PostsInfos;
}

function ProcessPost(&$PostInfo) //PostInfo must have fields id_msg, body, body_md5, and body_html
{
   //Load SMF
   global $ForumInfo;
   if(!defined('SMF'))
   {
       global $context;
       require_once(rtrim($ForumInfo['Location'], DIRECTORY_SEPARATOR).DIRECTORY_SEPARATOR.'SSI.php');
       mysql_select_db($ForumInfo['DBName']);
       SetEncoding();
   }

   //Update the cached body_html field
   $ParsedCode=$PostInfo['body_html']=parse_bbc($PostInfo['body']);
   $EscapedHTMLBody=mysql_escape_string($ParsedCode);
   $BodyMD5=md5($PostInfo['body']);
   mysql_query("UPDATE $ForumInfo[MessageTableName] SET body_html='$EscapedHTMLBody', body_md5='$BodyMD5' WHERE id_msg=$PostInfo[id_msg]");
}

//The fields to select in the Post query
function PostFields() { return Array('id_msg', 'poster_time', 'id_member', 'subject', 'poster_name', 'body', 'body_md5', 'body_html'); }

//Swap character encodings. Needs to be set to utf8
function SetEncoding($GetOld=false, $NewSet=Array('utf8', 'utf8', 'utf8'))
{
   //Get the old charset if required
   $CharacterVariables=Array('character_set_client', 'character_set_results', 'character_set_connection');
   $OldSet=Array();
   if($GetOld)
   {
       //Fill in variables with default in case they are not found
       foreach($CharacterVariables as $Index => $Variable)
           $OldSet[$Variable]='utf8';

       //Query for the character sets and update the OldSet array
       $Query=mysql_query('SHOW VARIABLES LIKE "character_%"');
       while($VariableInfo=mysql_fetch_assoc($Query))
           if(isset($OldSet[$VariableInfo['Variable_name']]))
               $OldSet[$VariableInfo['Variable_name']]=$VariableInfo['Value'];

       $OldSet=array_values($OldSet); //Turn back into numerical array
   }

   //Change to the new database encoding
   $CompiledSets=Array();
   foreach($CharacterVariables as $Index => $Variable)
       $CompiledSets[$Index]=$CharacterVariables[$Index].'="'.mysql_escape_string($NewSet[$Index]).'"';
   mysql_query('SET '.implode(', ', $CompiledSets));

   //If requested, return the previous values
   return $OldSet;
}
?>

78
Posts / Blacklisting DNS Server on Amazon EC2
« on: January 27, 2016, 07:16:17 pm »

Amazon EC2 is a great resource for cheap virtual servers to do simple things, like DNS or (low bandwidth) VPNs. I had the need this morning to set up a DNS server for a company which needed to blacklist a list of domains. The simplest way to do this is by editing all the computers’ hostfiles, but that method leaves a lot to be desired. Namely, blocking entire domains (as opposed to single subdomains), and deploying changes. Centralizing in a single place makes the job instant, immediate, and in the end, faster.

The following are the steps I used to set this up on an EC2 server. All command line instructions are followed by a single command you can run to execute the step. There is a full script below, at the end of the post, containing all steps from when you first login to SSH ("Login to root") to the end.


I am not going to go into the details of setting up an EC2 instance, as that information can be found elsewhere. I will also be skipping over some of the more obvious steps. Just create a default EC2 instance with the “Amazon Linux AMI”, and I will list all the changes that need to be made beyond that.

  • Creating the instance
    • For the first year, for the instance type, you might as well use a t2.micro, as it is free. After that, a t2.nano (which is a new lower level) currently at $56.94/year ($0.0065/Hour), should be fine.
    • After you select your instance type, click “Review and Launch” to launch the instance with all of the defaults.
    • After the confirmation screen, it will ask you to create a key pair. You can see other tutorials about this and how it enables you to log into your instance.
  • Edit the security group
    • Next, you need to edit the security group for your instance to allow incoming connections.
    • Go to “Instances” under the “Instances” group on the left menu, and click your instance.
    • In the bottom of the window, in the “Descriptions” tab, click the link next to “Security Groups”, which will bring you to the proper group in the security groups tab.
    • Right click it and “Edit inbound Rules”.
    • Make sure it has the following rules with Source=Anywhere: ALL ICMP [For pinging], SSH, HTTP, DNS (UDP), DNS (TCP)
  • Assign a permanent IP to your instance
    • To do this, click the “Elastic IPs” under “Network & Security” in the left menu.
    • Click “Allocate New Address”.
    • After creating it, right click the new address, then “Associate Address”, and assign it to your new instance.
  • You should probably set this IP up as an A record somewhere. I will refer to this IP as dns.yourdomain.com from now on.
  • Login to root
    • SSH into your instance as the ec2-user via “ssh ec2-user@dns.yourdomain.com”. If in windows, you could also use putty.
    • Sudo into root via “sudo su”.
  • Allow root login
    • At this point, I recommend setting it up so you can directly root into the server. Warning: some people consider this a security risk.
    • Copy your key pair(s) to the root user via “cat /home/ec2-user/.ssh/authorized_keys > /root/.ssh/authorized_keys
    • Set SSHD to permit root logins by changing the PermitRootLogin variable to “yes” in /etc/ssh/sshd_config. A quick command to do this is “perl -pi -e 's/^\s*#?\s*PermitRootLogin.*$/PermitRootLogin yes/igm' /etc/ssh/sshd_config”, and then reload the SSHD config with “service sshd reload”. Make sure to attempt to directly log into SSH as root before exiting your current session to make sure you haven’t locked yourself out.
  • Install apache (the web server), bind/named (the DNS server), and PHP (a scripting language)
    • yum -y install bind httpd php
  • Start and set services to run at boot
    • service httpd start; service named start; chkconfig httpd on; chkconfig named on;
  • Set the DNS server to be usable by other computers
    • Edit /etc/named.conf and change the 2 following lines to have the value “any”: “listen-on port 53” and “allow-query”
    • perl -pi -e 's/^(\s*(?:listen-on port 53|allow-query)\s*{).*$/$1 any; };/igm' /etc/named.conf; service named reload;
  • Point the DNS server to the blacklist files
    • This is done by adding “include "/var/named/blacklisted.conf";” to /etc/named.conf
    • echo -ne '\ninclude "/var/named/blacklisted.conf";' >> /etc/named.conf
  • Create the blacklist domain list file
    • touch /var/named/blacklisted.conf
  • Create the blacklist zone file
    • Put the following into /var/named/blacklisted.db . Make sure to change dns.yourdomain.com to your domain (or otherwise, “localhost”), and 1.1.1.1 to dns.yourdomain.com’s (your server’s) IP address. Make sure to keep all periods intact.

      $TTL 14400
      @       IN SOA dns.yourdomain.com. dns.yourdomain.com ( 2003052800  86400  300  604800  3600 )
      @       IN      NS   dns.yourdomain.com.
      @       IN      A    1.1.1.1
      *       IN      A    1.1.1.1
    • The first 2 lines tell the server the domains belong to it. The 3rd line sets the base blacklisted domain to your server’s IP. The 4th line sets all subdomains of the blacklisted domain to your server’s IP.
    • This can be done via (Update the first line with your values)

      YOURDOMAIN="dns.yourdomain.com"; YOURIP="1.1.1.1";
      echo -ne "\$TTL 14400\n@       IN SOA $YOURDOMAIN. $YOURDOMAIN ( 2003052800  86400  300  604800  3600 )\n@       IN      NS   $YOURDOMAIN.\n@       IN      A    $YOURIP\n*       IN      A    $YOURIP" > /var/named/blacklisted.db;
  • Fix the permissions on the blacklist files
    • chgrp named /var/named/blacklisted.*; chmod 660 /var/named/blacklisted.*;
  • Set the server’s domain resolution name servers
    • The server always needs to look at itself before other DNS servers. To do this, comment out everything in /etc/resolv.conf and add to it “nameserver localhost”. This is not the best solution. I’ll find something better later.
    • perl -pi -e 's/^(?!;)/;/gm' /etc/resolv.conf; echo -ne '\nnameserver localhost' >> /etc/resolv.conf
  • Run a test
    • At this point, it’s a good idea to make sure the DNS server is working as intended. So first, we’ll add an example domain to the DNS server.
    • Add the following to /var/named/blacklisted.conf and restart named to get the server going with example.com: “zone "example.com" { type master; file "blacklisted.db"; };
    • echo 'zone "example.com" { type master; file "blacklisted.db"; };' >> /var/named/blacklisted.conf; service named reload;
    • Ping “test.example.com” and make sure it’s IP is your server’s IP
    • Set your computer’s DNS to your server’s IP in your computer’s network settings, ping “test.example.com” from your computer, and make sure the returned IP is your server’s IP. If it works, you can restore your computer’s DNS settings.
  • Have the server return a message when a blacklisted domain is accessed
    • Add your message to /var/www/html
    • echo 'Domain is blocked' > /var/www/html/index.html
    • Set all URL paths to show the message by adding the following to the /var/www/html/.htaccess file

      RewriteEngine on
      RewriteCond %{REQUEST_URI} !index.html
      RewriteCond %{REQUEST_URI} !AddRules/
      RewriteRule ^(.*)$ /index.html [L]
    • echo -ne 'RewriteEngine on\nRewriteCond %{REQUEST_URI} !index.html\nRewriteCond %{REQUEST_URI} !AddRules/\nRewriteRule ^(.*)$ /index.html [L]' > /var/www/html/.htaccess
    • Turn on AllowOverride in the /etc/httpd/conf/httpd.conf for the document directory (/var/www/html/) via “ perl -0777 -pi -e 's~(<Directory "/var/www/html">.*?\n\s*AllowOverride).*?\n~$1 All~s' /etc/httpd/conf/httpd.conf
    • Start the server via “service httpd graceful
  • Create a script that allows apache to refresh the name server’s settings
    • Create a script at /var/www/html/AddRules/restart_named with “/sbin/service named reload” and set it to executable
    • mkdir /var/www/html/AddRules; echo '/sbin/service named reload' > /var/www/html/AddRules/restart_named; chmod 755 /var/www/html/AddRules/restart_named
    • Allow the user to run the script as root by adding to /etc/sudoers “apache ALL=(root) NOPASSWD: /var/www/html/AddRules/restart_named” and “Defaults!/var/www/html/AddRules/restart_named !requiretty
    • echo -e 'apache ALL=(root) NOPASSWD:/var/www/html/AddRules/restart_named\nDefaults!/var/www/html/AddRules/restart_named !requiretty' >> /etc/sudoers
  • Create a script that allows the user to add, remove, and list the blacklisted domains
    • Add the following to /var/www/html/AddRules/index.php (one line command not given. You can use “nano” to create it)
      <?php
      //Get old domains
      $BlockedFile='/var/named/blacklisted.conf';
      $CurrentZones=Array();
      foreach(explode("\n", file_get_contents($BlockedFile)) as $Line)
             if(preg_match('/^zone "([\w\._-]+)"/', $Line, $Results))
                     $CurrentZones[]=$Results[1];

      //List domains
      if(isset($_REQUEST['List']))
             return print implode('
      '
      , $CurrentZones);


      //Get new domains
      if(!isset($_REQUEST['Domains']))
             return print 'Missing Domains';
      $Domains=$_REQUEST['Domains'];
      if(!preg_match('/^[\w\._-]+(,[\w\._-]+)*$/uD', $Domains))
             return print 'Invalid domains string';
      $Domains=explode(',', $Domains);

      //Remove domains
      if(isset($_REQUEST['Remove']))
      {
             $CurrentZones=array_flip($CurrentZones);
             foreach($Domains as $Domain)
                     unset($CurrentZones[$Domain]);
             $FinalDomainList=array_keys($CurrentZones);
      }
      else //Combine domains
             $FinalDomainList=array_unique(array_merge($Domains, $CurrentZones));

      //Output to the file
      $FinalDomainData=Array();
      foreach($FinalDomainList as $Domain)
             $FinalDomainData[]=
                     
      "zone \"$Domain\" { type master; file \"blacklisted.db\"; };";

      file_put_contents($BlockedFile, implode("\n", $FinalDomainData));

      //Reload named
      print `sudo /var/www/html/AddRules/restart_named`;
      ?>
    • Add the “apache” user to the “named” group so the script can update the list of domains in /var/named/blacklisted.conf via “usermod -a -G named apache; service httpd graceful;
  • Run the domain update script
    • To add a domain (separate by commas): http://dns.yourdomain.com/AddRules/?Domains=domain1.com,domain2.com
    • To remove a domain (add “Remove&” after the “?”): http://dns.yourdomain.com/AddRules/?Remove&Domains=domain1.com,domain2.com
    • To list the domains: http://dns.yourdomain.com/AddRules/?List
  • Password protect the domain update script
    • Add to AddRules/.htaccess the following

      AuthType Basic
      AuthName "Admins Only"
      AuthUserFile "/var/www/html/AddRules/.htpasswd"
      require valid-user
    • echo -ne 'AuthType Basic\nAuthName "Admins Only"\nAuthUserFile "/var/www/html/AddRules/.htpasswd"\nrequire valid-user' > /var/www/html/AddRules/.htaccess
    • Warning: Putting the password file in an http accessible directory is a security risk. I just did this for sake of organization.
    • Create the user+password via “htpasswd -bc /var/www/html/AddRules/.htpasswd USERNAME” and then entering the password


[Edit on 2016-01-30 @ noon]

To permanently set “localhost” as the resolver DNS, add “DNS1=localhost” to “/etc/sysconfig/network-scripts/ifcfg-eth0”. I have not yet confirmed this edit.

Security Issue

Soon after setting up this DNS server, it started getting hit by a DNS amplification attack. As the server is being used as a client’s DNS server, turning off recursion is not available. The best solution is to limit the people who can query the name server via an access list (usually a specific subnet), but that would very often not be an option either. The solution I currently have in place, which I have not actually verified if it works, is to add a forced-forward rule which only makes external requests to the name server provided by Amazon. To do this, get the name server’s IP from /etc/resolv.conf (it should be commented from an earlier step). Then add the following to your named.conf in the “options” section.


   forwarders {
      DNS_SERVER_IP;
   };
   forward only;

After I added this rule, external DNS requests stopped going through completely. To fix this, I turned “dnssec-validation” to “no” in the named.conf. Don’t forget to restart the service once you have made your changes.

[End of edit]

Full serverside script
Make sure to run this as root (login as root or sudo it)

Download the script here. Make sure to chmod and sudo it when running. “chmod +x dnsblacklist_install.sh; sudo ./dnsblacklist_install.sh;

#User defined variables
VARIABLES_SET=0; #Set this to 1 to allow the script to run
YOUR_DOMAIN="localhost";
YOUR_IP="1.1.1.1";
BLOCKED_ERROR_MESSAGE="Domain is blocked";
ADDRULES_USERNAME="YourUserName";
ADDRULES_PASSWORD="YourPassword";


#Confirm script is ready to run
if [ $VARIABLES_SET != 1 ]; then
   echo 'Variables need to be set in the script';
   exit 1;
fi
if [ `whoami` != 'root' ]; then
   echo 'Must be root to run script. When running the script, add "sudo" before it to' \
       'run as root'
;
   exit 1;
fi

#Allow root login
cat /home/ec2-user/.ssh/authorized_keys > /root/.ssh/authorized_keys;
perl -pi -e 's/^\s*#?\s*PermitRootLogin.*$/PermitRootLogin yes/igm' /etc/ssh/sshd_config;
service sshd reload;

#Install services
yum -y install bind httpd php;
chkconfig httpd on;
chkconfig named on;
service httpd start;
service named start;

#Set the DNS server to be usable by other computers
perl -pi -e 's/^(\s*(?:listen-on port 53|allow-query)\s*{).*$/$1 any; };/igm' \
   /etc/named.conf;
service named reload;

#Create/link the blacklist files
echo -ne '\ninclude "/var/named/blacklisted.conf";' >> /etc/named.conf;
touch /var/named/blacklisted.conf;

#Create the blacklist zone file
echo -ne "\$TTL 14400
@       IN SOA $YOUR_DOMAIN. $YOUR_DOMAIN ( 2003052800  86400  300  604800  3600 )
@       IN      NS   $YOUR_DOMAIN.
@       IN      A    $YOUR_IP
*       IN      A    $YOUR_IP" > /var/named/blacklisted.db;

#Fix the permissions on the blacklist files
chgrp named /var/named/blacklisted.*;
chmod 660 /var/named/blacklisted.*;

#Set the server’s domain resolution name servers
perl -pi -e 's/^(?!;)/;/gm' /etc/resolv.conf;
echo -ne '\nnameserver localhost' >> /etc/resolv.conf;

#Run a test
echo 'zone "example.com" { type master; file "blacklisted.db"; };' >> \
   /var/named/blacklisted.conf;
service named reload;
FOUND_IP=`dig -t A example.com | grep -ioP "^example\.com\..*?"'in\s+a\s+[\d\.:]+' | \
   
grep -oP '[\d\.:]+$'`
;
if [ "$YOUR_IP" == "$FOUND_IP" ]
then
 echo 'Success: Example domain matches your given IP' > /dev/stderr;
else
 echo 'Warning: Example domain does not match your given IP' > /dev/stderr;
fi

#Have the server return a message when a blacklisted domain is accessed
echo "$BLOCKED_ERROR_MESSAGE" > /var/www/html/index.html;
perl -0777 -pi -e 's~(<Directory "/var/www/html">.*?\n\s*AllowOverride).*?\n~$1 All~s' \
   
/etc/httpd/conf/httpd.conf;
echo -n 'RewriteEngine on
RewriteCond %{REQUEST_URI} !index.html
RewriteCond %{REQUEST_URI} !AddRules/
RewriteRule ^(.*)$ /index.html [L]' > /var/www/html/.htaccess;
service httpd graceful;

#Create a script that allows apache to refresh the name server’s settings
mkdir /var/www/html/AddRules;
echo '/sbin/service named reload' > /var/www/html/AddRules/restart_named;
chmod 755 /var/www/html/AddRules/restart_named;

echo 'apache ALL=(root) NOPASSWD:/var/www/html/AddRules/restart_named
Defaults!/var/www/html/AddRules/restart_named !requiretty' >> /etc/sudoers;

#Create a script that allows the user to add, remove, and list the blacklisted domains
echo -n $'<?php
//Get old domains
$BlockedFile=\'/var/named/blacklisted.conf\';
$CurrentZones=Array();
foreach(explode("\\n", file_get_contents($BlockedFile)) as $Line)
       if(preg_match(\'/^zone "([\\w\\._-]+)"/\', $Line, $Results))
               $CurrentZones[]=$Results[1];

//List domains
if(isset($_REQUEST[\'List\']))
       return print implode(\'
\', $CurrentZones);


//Get new domains
if(!isset($_REQUEST[\'Domains\']))
       return print \'Missing Domains\';
$Domains=$_REQUEST[\'Domains\'];
if(!preg_match(\'/^[\\w\\._-]+(,[\\w\\._-]+)*$/uD\', $Domains))
       return print \'Invalid domains string\';
$Domains=explode(\',\', $Domains);

//Remove domains
if(isset($_REQUEST[\'Remove\']))
{
       $CurrentZones=array_flip($CurrentZones);
       foreach($Domains as $Domain)
               unset($CurrentZones[$Domain]);
       $FinalDomainList=array_keys($CurrentZones);
}
else //Combine domains
       $FinalDomainList=array_unique(array_merge($Domains, $CurrentZones));

//Output to the file
$FinalDomainData=Array();
foreach($FinalDomainList as $Domain)
   $FinalDomainData[]="zone \\"$Domain\\" { type master; file \\"blacklisted.db\\"; };";
file_put_contents($BlockedFile, implode("\\n", $FinalDomainData));

//Reload named
print `sudo /var/www/html/AddRules/restart_named`;
?>' > /var/www/html/AddRules/index.php;

usermod -a -G named apache;
service httpd graceful;

#Password protect the domain update script
echo -n 'AuthType Basic
AuthName "Admins Only"
AuthUserFile "/var/www/html/AddRules/.htpasswd"
require valid-user' > /var/www/html/AddRules/.htaccess;

htpasswd -bc /var/www/html/AddRules/.htpasswd "$ADDRULES_USERNAME" "$ADDRULES_PASSWORD";

echo 'Script complete';

79
Posts / Re: Useful Bash commands and scripts
« on: January 25, 2016, 10:09:29 pm »
Clear the buffer of a terminal in bash
Code: [Select]
echo -e '\0033\0143'

80
Posts / Re: Useful Bash commands and scripts
« on: January 25, 2016, 09:48:13 pm »
To get the exported entries from a single dll in cygwin, create a script with the following code. It takes 1 argument as a parameter.
Code: [Select]
objdump -p $1 | grep -Pzo '(?is)^\[Ordinal/Name Pointer\] Table.*?\n\n' | grep -oP '(?<=\d\] ).*$'If you saved the script as "get_dll_exports", to run it against multiple DLLs at a time, create another script as follows
Code: [Select]
for i in $@; do echo -e "--------\n$i\n--------"; get_dll_exports $i; doneOr to process multiple dlls, but output all of a single file's results on one line with the filename preceding
Code: [Select]
for i in $@; do echo -n "$i: "; get_dll_exports $i | perl -pe 's/\n/ /' -; echo; done

81
Updates / LetsEncrypt Better Apache Installer v1.0
« on: January 16, 2016, 11:52:12 pm »

Released v1.0 of LetsEncrypt Better Apache Installer, which “Installs SSL/HTTPS certificates via letsencrypt for all domains”.

82
Projects / LetsEncrypt Better Apache Installer
« on: January 16, 2016, 11:21:46 pm »

Description: Installs SSL/HTTPS certificates via letsencrypt for all domains.
Information: Default configuration is for cPanel.
  • This takes a single VirtualHost domain and will install certificates for all VirtualHosts on the same IP (or if requested, just the given VirtualHost)
  • This script can be run through both a bash command line (CLI), and as a web page. Parameter names use a different format for the two
  • While this script was originally designed for cPanel, it should work with any apache configuration, given the correct parameters
Languages: PHP

83
Updates / Updated windows ln for cygwin script
« on: January 12, 2016, 01:54:55 am »

84
Posts / Syncing Amazon EC2 Instances
« on: December 10, 2015, 09:47:10 pm »
Original post for Syncing Amazon EC2 Instances can be found at https://www.castledragmire.com/Posts/Syncing_Amazon_EC2_Instances.
Originally posted on: 12/10/15

In continuation of yesterday’s post, in which I showed how to create Amazon AMIs to keep your newly created EC2 instances up to date, today I will cover syncing already-live instances from the master to slaves. All of the below takes place on the master instance, and assumes all other instances are part of the slave group. You may have to use extra filters on the below “aws” command to only pull IPs from a certain group of instances.

Here is a simple bash script (hereby referred to as “Propagate.sh”) which syncs /var/www/html/ to all of your slave instances. It uses the “aws” command line interface provided by Amazon, which comes default with the Amazon Linux starter AMI.


#The first command line of the script contains the master’s IP, so it does not sync with itself.
export LocalIP=Your_Master_IP_Here;

#Get the IPs of all slave instances
export NewIPs=`aws ec2 describe-instances | grep '"PrivateIpAddress"' | perl -i -pe 's/(^.*?: "|",?\s*?$)//gm' | sort -u | grep -v $LocalIP`

#Loop over all slave instances
for i in $NewIPs; do
       echo "Syncing to: $i";
       #Run an rsync from the master to the slave
       rsync -aP -e 'ssh -o StrictHostKeyChecking=no' /var/www/html/ root@$i:/var/www/html/;
done

You may also want to add “-o UserKnownHostsFile=/dev/null” to the SSH command (directly after “-o StrictHostKeyChecking=no”), as a second EC2 instance may end up having the same IP as a previously terminated instance. Another solution to that problem is syncing the “/etc/ssh/ssh_host_rsa_key*” from the master when an instance initializes, so all instances keep the same SSH fingerprint.


To let other people manually execute this script, you can create a PHP file with the following in it. (Change /var/www/ in all below examples to where you place your Propagate.sh)

<? print nl2br(htmlentities(shell_exec('sudo /var/www/Propagate.sh 2<&1'))); ?>

If your Propagate.sh needs to be ran as root, which it may if your PHP environment is not run as the user root (usually “apache”), then you need to make sure it CAN run as root without intervention. To do this, add the following to the /etc/sudoers file
apache  ALL=(ALL)       NOPASSWD: /usr/bin/whoami, /var/www/Propagate.sh
Change the user from “apache” to the user which PHP runs as (when running through apache).
I included “whoami” as a valid sudoer application for testing purposes.
Also, in the sudoers file, if “Defaults requiretty” is turned on, you will need to comment it/turn it off.

While I did not mention it in yesterday's post, I thought I should at least mention it here. There are other ways to keep file systems in sync with each other. This is just a good use case for when you want to keep all instances as separate independent entities. Another solution to many of the previously mentioned problems is using Amazon's new EFS, which is currently still in preview mode.


85
Posts / Custom Initializations for Amazon AMIs
« on: December 09, 2015, 04:26:35 pm »

I was recently hired to move a client's site from our primary server in Houston to the Amazon cloud, as it was about to take a big hit in traffic. The normal setup for this kind of job is pretty straightforward. Move the database over to RDS, set up an AMI of an EC2 instance, a load balancer, and ec2 auto scaling. However, there were a couple of problems I needed to solve this time around for the instances launched via the auto scalar that I had not really needed to do before. This includes syncing the SSH settings and current codebase from the primary instance, as opposed to recreating AMIs every time there was a change. So, long story short, here are the problems and solutions that need to be added before the AMI image is created.


This all assumes you are running as root. Most of these commands should work on any Linux distribution that Amazon has default AMIs for, but some of these may only work in the Amazon and CentOS AMIs.


Pre-setup:
  • Your first instance that you are creating the AMI from should be a permanent instance. This is important for 2 reasons.
    1. When changing configurations for the auto scalar, if and when your instances are terminated and recreated, this instance will always be available on the load balancer, so there is no downtime.
    2. This instance can act as a central repository for other instances to sync from.
    So make sure this instance has an elastic IP assigned to it. From here on out, we will refer to this instance as PrimaryInstance (you can set this physically in the host file, or change it in all scripts to however you want to refer to your elastic IP [most likely through a DNS domain]).
  • Create your ssh private key for the instances: (For all prompts, use default settings)
    ssh-keygen -t rsa -b 4096 -C "your_email@example.com"
  • Make sure your current ssh authorized_keys contains your new ssh private key:
    cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
  • Make sure your ssh known_hosts includes your primary instance, so all future ssh calls to it are automatically accept it as a known host:
    ssh PrimaryInstance -o StrictHostKeyChecking=no
    You do not have to finish the login process. This just makes sure our primary instance will be recognized by other instances.
  • Turn on PermitRootLogin in /etc/ssh/sshd_config and reload the sshd config service sshd reload
    I just recommend this because it makes life way, way easier. The scripts below assume that you did this.

Create a custom init file that runs on boot to take care of all the commands that need to be run.

#Create the script and make sure the full path (+all other root environment variables) are set when it is ran
echo '#!/bin/bash -l' > /etc/rc.d/init.d/custom_init

#Set the script as executable
chmod +x /etc/rc.d/init.d/custom_init

#Executes it as one of the last scripts on run level 3 (Multi-user mode with networking)
ln -s ../init.d/custom_init /etc/rc.d/rc3.d/S99custom_init
All of the below commands in this post will go into this script.

Allow login via password authentication:

perl -i -pe 's/^PasswordAuthentication.*$/PasswordAuthentication yes/mg' /etc/ssh/sshd_config
service sshd reload
Notes:
You may not want to do this. It was just required by my client in this case.
This is required in the startup script because Amazon likes to mess with the sshd_config (and authorized_keys) in new instances it boots.

Sync SSH settings from the PrimaryInstance:

#Remove the known_hosts file, in case something on the PrimaryInstance has changed that would block ssh commands.
rm -f ~/.ssh/known_hosts

#Sync the SSH settings from the PrimaryInstance
rsync -e 'ssh -o StrictHostKeyChecking=no' -a root@PrimaryInstance:~/.ssh/ ~/.ssh/

Sync required files from the PrimaryInstance. In this case, the default web root folder:
rsync -at root@PrimaryInstance:/var/www/html/ /var/www/html/

That's it for the things that need to be configured/added to the instance. From there, create your AMI and launch config, and create/modify your launch group and load balancer.


Also, as a very important note about your load balancer, make sure if you are mirroring its IP on another domain to use a CNAME record, and not the IP in an A record, as the load balancer IP is subject to change.


86
Posts / Lets Encrypt HTTPS Certificates
« on: December 03, 2015, 11:12:54 pm »
Original post for Lets Encrypt HTTPS Certificates can be found at https://www.castledragmire.com/Posts/Lets_Encrypt_HTTPS_Certificates.
Originally posted on: 12/03/15

After a little over a year of waiting, Let’s Encrypt has finally opened its doors to the public! Let’s Encrypt is a free https certificate authority, with the goal of getting the entire web off of http (unencrypted) and on to https. I consider this a very important undertaking, as encryption is one of the best ways we can fight illegal government surveillance. The more out there that is encrypted, the harder it will be to spy on people.

I went ahead and got it up and running on 2 servers today, which was a bit of a pain in the butt. It [no longer] supports Python 2.6, and was also very unhappy with my CentOS 6.4 cPanel install. Also, when you first run the letsencrypt-auto executable script as instructed by the site, it opens up your package manager and immediately starts downloading LOTS of packages. I found this to be quite anti-social, especially as I had not yet seen anywhere, or been warned, that it would do this before I started the install, but oh well. It is convenient. The problem in cPanel was that a specific library, libffi, was causing problems during the install.


To fix the Python problem for all of my servers, I had to install Python 2.7 as an alt Python install so it wouldn’t mess with any existing infrastructure using Python 2.6. After that, I also set the current alias of “python” to “python2.7” so the local shell would pick up on the correct version of Python.


As root in a clean directory:
wget https://www.python.org/ftp/python/2.7.8/Python-2.7.8.tgz
tar -xzvf Python-2.7.8.tgz
cd Python-2.7.8
./configure --prefix=/usr/local
make
make altinstall
alias python=python2.7

The cPanel lib problem was caused by libffi already being installed as 3.0.9-1.el5.rf, but yum wanted to install its devel package as version 3.0.5-3.2.el6.x86_64 (an older version). It did not like running conflicting versions. All that was needed to fix the problem was to manually download and install the same devel version as the current live version.

wget http://pkgs.repoforge.org/libffi/libffi-devel-3.0.9-1.el5.rf.x86_64.rpm
rpm -ivh libffi-devel-3.0.9-1.el5.rf.x86_64.rpm

Unfortunately, the apache plugin was also not working, so I had to do a manual install with “certonly” and “--webroot”.


And that was it; letsencrypt was ready to go and start signing my domains! You can check out my current certificate, issued today, that currently has 13 domains tied to it!


87
Posts / PHPMyAdmin SQL Export: Key Position
« on: December 01, 2015, 11:33:23 pm »

After version 4.2.0.0 (2014-05-08) of phpMyAdmin, it stopped including table’s keys inline within the create table statement, and instead opted to add all the table keys at the very end of the export file by modifying the tables. (See "rfe #1004 Create indexes at the end in SQL export). This behavior has been annoying to many people, including myself, but I never noticed anyone mentioning a fix. I looked into the source and there is a very simple way to restore this behavior to what it originally was.


Edit the file “phpMyAdmin/libraries/plugins/export/ExportSql.class.php”. In it, the code block starting with the below line needs to be skipped
if (preg_match('@CONSTRAINT|KEY@', $create_query)) {
The easiest way to do this is changing that line to
if (false && preg_match('@CONSTRAINT|KEY@', $create_query)) {

88
Posts / AutoHotKey Scripts
« on: December 01, 2015, 09:29:01 pm »
Original post for AutoHotKey Scripts can be found at https://www.castledragmire.com/Posts/AutoHotKey_Scripts.
Originally posted on: 12/01/15

In lieu of using my own custom C++ background services to take care of hot key tasks in Windows, I started using AutoHotKey a while back. While it’s not perfect, and it is missing a lot of Win32 API functionality, I am still able to mostly accomplish what I want in it. I was thinking I should add some of the simple scripts I use here.


Center a string within padding characters and output as key-strokes
Example:
  • PadText = ~*
  • Length = 43
  • Text = Example Text
  • Result = ~*~*~*~*~*~*~*~*Example Text~*~*~*~*~*~*~*~

;Get the last values
IniPath=%A_ScriptDir%\AutoHotKey.ini
IniRead,PadText,%IniPath%,CenterString,PadText,-
IniRead,NewLength,%IniPath%,CenterString,NewLength,10
IniRead,TheString,%IniPath%,CenterString,TheString,The String

;Get the input
InputBox,PadText,Center String,Pad Character,,,,,,,,%PadText%
InputBox,NewLength,Center String,New Length,,,,,,,,%NewLength%
InputBox,TheString,Center String,String To Center,,,,,,,,%TheString%

;Cancel on blank pad or invalid number
if StrLen(PadText)==0
{
   MsgBox,Pad text cannot be blank
   return
}
if NewLength is not integer
{
   MsgBox,New length must be an integer
   return
}

;Save the last values
IniWrite,%PadText%,%IniPath%,CenterString,PadText
IniWrite,%NewLength%,%IniPath%,CenterString,NewLength
IniWrite,%TheString%,%IniPath%,CenterString,TheString

;Initial padding
PadStrLen:=StrLen(PadText)
PadLen:=NewLength-StrLen(TheString)
NewString:=""
Loop
{
   if StrLen(NewString)>=Ceil(PadLen/2)
      break
   NewString.=PadText
}

;Truncate initial padding to at least half
NewString:=Substr(NewString, 1, Ceil(PadLen/2))

;Add the string
NewString.=TheString

;Final padding
Loop
{
   if StrLen(NewString)>=NewLength
      break
   NewString.=PadText
}

;Truncate to proper length
NewString:=Substr(NewString, 1, NewLength)

;Output to console
Sleep,100
Send %NewString%
return

Format rich clipboard text to plain text

clipboard = %clipboard%
return

Force window to borderless full screen
Description: This takes the active window, removes all window dressing (titlebar, borders, etc), sets its resolution as 1920x1080, and positions the window at 0x0. In other words, this makes your current window take up the entirety of your primary monitor (assuming it has a resolution of 1920x1080).

WinGetActiveTitle, WinTitle
WinSet, Style, -0xC40000, %WinTitle%
WinMove, %WinTitle%, , 0, 0, 1920, 1080
return

Continually press key on current window
Description: Saves the currently active window (by its title) and focused control object within the window; asks the user for a keypress interval and the key to press; starts to continually press the requested key at the requested interval in the original control (or top level window if an active control is not found); stops via the F11 key.
Note: I had created this to help me get through the LISA intro multiple times.

;Get the current window and control
WinGetActiveTitle, TheTitle
ControlGetFocus FocusedControl, %TheTitle%
if(ErrorLevel)
   FocusedControl=ahk_parent

;Get the pause interval
InputBox,IntervalTime,Starting script with window '%TheTitle%',Enter pause interval in milliseconds. After submitted`, hold down the key to repeat,,,,,,,,200
if(ErrorLevel || IntervalTime=="") ;Cancel action if blank or cancelled
   return
IntervalTime := IntervalTime+0

;Get the key to keep pressing - Unfortunately, there is no other way I can find to get the currently pressed keycode besides polling all 255 of them
Sleep 500 ;Barrier to make sure one of the initialization keys is not grabbed
Loop {
   TestKey := 0
   Loop {
      SetFormat, INTEGER, H
      HexTextKey := TestKey
      SetFormat, INTEGER, D
      VirtKey = % "vk" . SubStr(HexTextKey, 3)
      if(GetKeyState(VirtKey)=1 || TestKey>255)
         break
      TestKey:=TestKey+1
   }
   if(TestKey<=255)
      break
   Sleep 500
}
VirtKey := GetKeyName(VirtKey)

;If a direction key, remap to the actual key
if(TestKey>=0x25 && TestKey<=0x28)
   VirtKey := SubStr(VirtKey, 7)

;Let the user know their key
MsgBox Received key: '%VirtKey%'. You may now let go of the key. Hold F11 to stop the script.

;Continually send the key at the requested interval
KeyDelay:=10
SetKeyDelay %KeyDelay% #Interval between up/down keys
IntervalTime-=%KeyDelay%
Loop {
   ;Press the key
   ControlSend, %FocusedControl%, {%VirtKey% Up}{%VirtKey% Down}, %TheTitle%

   ;Check for the cancel key
   if(GetKeyState("F11"))
      break

   ;Wait the requested interval to press the key again
   Sleep, %IntervalTime%
}

;Let the user know the script has ended
MsgBox Ending script with window '%TheTitle%'
return

89
Projects / Re: HackPics
« on: December 01, 2015, 08:51:28 pm »
Ah, ok. The assumption that you had deleted the posts was a very large part of it. I had assumed you had just come in, "demanded" something, and then when I gave you an answer that made you realize your original statements were completely wrong, you had deleted them and decided to hide that you asked it. I didn't add that "Edit" note until a week or more later, when you had not responded, which I found to be unlikely as innocent coincidence since you had been asking once or more a week. It is very believable that the posts were somehow deleted by the forum. I'm doing better now about keeping backups.

90
Posts / LISA game difficulty level save hack
« on: December 01, 2015, 08:43:20 pm »

I recently bought the game LISA on Steam, and the humor approach is fascinating. Unfortunately, this approach involves being incredible vague, or outright obtuse, at telling you what is going on, or what is going to happen if you do something. The very first choice you have in the game is whether to choose “Pain” mode or “Normal” mode. It doesn’t tell you anything beyond that. Unfortunately, I interpreted this as “Normal” and “Easy”, and so I chose the former “Pain” mode. One of the “features” of pain mode is that you can only use save points once, and there are only 36 of them in the game, spread very far apart. After I was a few hours into the game, and I realized how much of a bother this was going to be, especially because it meant I had to play in possible multi-hour chunks, not knowing when I would get to stop. I didn’t feel like replaying up until that point, so I decided to do some save game file hacking, as that is part of the fun for me.

DO NOTE, this method involves deleting some of the data in the game file, specifically a bunch of boolean flags, which might cause some events in the save to be “forgotten”, so they will reoccur. At the point of the game I was at, the few deleted flag actions that I encountered didn’t affect anything big or of importance. One example of this is the long-winded character repeats his final soliloquy when you enter his map.


So, to switch from “Pain” mode to “Normal” mode in the save file, do the following:
  1. Your save files are located at %STEAM_FOLDER%/steamapps/common/LISA/Save##.rvdata2
  2. Backup the specific save file you want to edit, just in case.
  3. Open that save file in a hex editor. You might need to be in steam offline mode for the edit to stick.
  4. Search for “@data[”. Immediately following it are the hex character “02 02 02”. Delete them and in their place, add the hex character 0x73 (“s”).
  5. Following the “s” character that you just added are 514 bytes that are either “0”, “T”, or “F”, and then a colon (“:”)
  6. Keep the first 110 of these bytes, and then delete everything up to the colon (which should be 404 bytes).
  7. Save the file, and that should be it!

Pages: 1 ... 4 5 [6] 7 8 ... 36