Interspire Miscellaneous Blog

Here's a quick shell command to work out where your disk space is being used.

du -s * | sort -n | tail

If you run this in a directory it will print out the top files and directories under the current directory using up space. For example, yesterday I started playing with a Fuzzing tool for testing shopping cart. Unfortunately Xdebug was setup to save execution traces and profiling files on the server. The end result after about an hour of fuzzing was a directory with about 67 Gigs worth of files in it. That command let me track down the cause of the hard drive filling up quickly.

What is spam?

Spam is the unsolicited sending of electronic mail to anyone. This can be to individual addresses or to an entire email marketing list consisting of millions of contacts. Research has determined that over 50% of all email transactions taken place in the United States are considered to be spam. This is a very serious issue because not only does it slow down our days clearing out our inbox's of spam (potentially missing and deleting valid emails) it is also placing a lot of overhead on the network system as a whole.

With all the complaints about spam, receiving it, having your own valid emails picked up as it, worrying about the legal ramifications on it, I thought it a good idea to do some research for you and find out exactly what the deal was and what we can do to avoid being marked as a spammer and potentially having our server black listed.

Step up the CAN SPAM Act 2003

In an attempt to try to control spamming in some way the Federal Trade Commission of America created the CAN SPAM Act. This Act is quite simple in its nature and easy for you to abide by. It does not stop you from sending unsolicited email. It simply tries to regulate it.

What this means is that you can still send emails to people that have not explicitly signed up to your mailing list but you must abide by the laid out rules in order to avoid the penalties.

What are these rules?

The rules are simple and very easy to follow:

• Do not set false or misleading header information.
  • That is your 'From' address, domain name etc must be correct. You cannot say you are from www.abc.com when you are in fact from www.xyz.com.
  • This does not mean that you always have to send from the same domain that Interspire Email Marketer is located on, it just means that the domain and person that you are saying you are must be real. So you can still send on your clients behalf from your own installation, just make sure they are real.
• Do not set false or misleading subject lines
  • This is simple. Make sure that your emails subject line and the content of the email are one and the same. Do not try to trick your contacts into opening the email.
  • This does not mean that you are not allowed to be creative with your subject lines. They are the first point of contact with your contacts and as such must be catchy and make your contacts want to open the email. But being clever with your subject lines and being deceitful are completly different matters.
• Give your contacts an opt out method
  • Once again, this is a pretty simple step. Make sure that you include in every email that you send out a way to unsubscribe.
  • Interspire Email Marketer provides you with 2 different methods to do this. The first is simply including the unsubscribe link (%%unsubscribelink%%) in your email. The second method is to create an unsubscribe form and place this on your website. From there you can link to the form from your email campaigns. Both methods are equally as valid.
• You must identify yourself and your company clearly and let the contact know that this email is an advertisement.
  • Your email campaign needs to clearly inform your contact that you are trying to sell them something (if you are) and provide them with your companies name and a valid physical postal address.

What penalties can you expect for these?

"Each violation of the above provisions is subject to fines of up to $11,000. Deceptive commercial email also is subject to laws banning false or misleading advertising."

What this means is that for every rule that you break in relation to this Act you could potentially be liable for $11,000. If you are found guilty of not allowing subscribers to unsubscribe as well as falsifying your from address so that your subscribers cannot get in touch with you then you could be liable for up to $22,000 in fines.

What does all this mean to me?

This can sound pretty full on but remember, if you do the right thing then you have nothing to worry about. Make sure you always have your details in your emails (Place them in your footer to be included), make sure you are honest with your contacts and you will be fine.

If you have any questions or comments at all about this please don't hesitate to post a comment bellow and I will get back to you.

Profiling your application

Profiling your application can be difficult to do. You can either do it by guesswork (not a very efficient way of doing it), or by using some tools to help you. Sometimes you'll be surprised at where a bottleneck occurs, which is why guessing doesn't work - you could be looking in the completely wrong spot!

One of the best tools when working with a PHP application is XDebug. I'm assuming you have root access to your server and you are working on a development machine - don't try this on in a production environment ;)

Installing it is quite easy, the instructions on their website are pretty straight forward and can be foundhere.

Once it's installed, you need to enable it in your php.ini file. To do that, open it up and jump to the bottom. Add the following line:

zend_extension=/path/to/xdebug.so

Where /path/to/xdebug.so is where it was installed.

If you are doing this for web applications, you'll need to restart your webserver. You'll also need to put the following in a .htaccess file so profiling is enabled for your app:

xdebug.profiler_enable=1
xdebug.profiler_output_dir=/tmp
xdebug.profiler_output_name=cachegrind.out.%t.%p

For command line scripts, place those directives in the php.ini file (after loading the zend_extension), they can't be done on the fly.

Once you have that going (check your web server logs for errors if necessary), you can fire up your application.

After you view a page, you'll see a file in the /tmp folder called 'cachegrind.out.timestamp.pid', eg 'cachegrind.out.1211853650.6921'. The timestamp & pid will change per page view, so if you're on a busy or shared development server, change the profile_output_dir to another writable folder (eg your application has a writable cache/ folder).

Now you have a profile script, how do you use it?

The best tool to do that is KCacheGrind, however that is for Linux based systems only. If you're on a Windows desktop, you can use WinCacheGrind - however it hasn't been updated in a while and doesn't show as much information as KCacheGrind.

Here's a screenshot of part of the profile of running an Email Marketer cron job (which sent an email to 50,000 subscribers while the app was in "test mode"). The graph tells us that most of the time is spent in the SS_Email_API->Send method. Inside that, most of the time is spent in _ReplaceCustomFields (which in turn shows most of the time is spent doing the replacements).

Using this method of profiling, we were able to find a problem with sending an email campaign with lots of trackable links. We significantly improved the way that link tracking occurred to enhance sending speed and still provide the same functionality.

Say you are just about ready to do a release of your project and you want to work out what files have changed for creating a changelog. Or maybe you want to have all the files in 1 directory for creating a zip to send to someone, or to copy over a test copy to test upgrading etc.

Using the below command you just need to change the two revision numbers (1403 and 1438 in the example) and the path to your repository (http://server/svn/project/trunk in the example below) and the command will export all the files that have changed between the two revisions.

I've only tested the command in bash but it should work in zsh too. Not sure about any other shells. Also make sure that you have the trailing / in the sed part of the command otherwise it won't work properly.

for i in $(svn diff --summarize -r 1403:1438 http://server/svn/project/trunk | awk '{ print $2 }'); do p=$(echo $i | sed -e 's{http://server/svn/project/trunk/{{'); mkdir -p $(dirname $p); svn export $i $p; done

If you just wanted to get a list of the changed files you could run the command

svn diff --summarize -r 1403:1438 http://server/svn/project/trunk | awk '{ print $2 }' |  sed -e 's{http://server/svn/project/trunk/{{'