“Geary is a new email reader for GNOME designed to let you read your email quickly and effortlessly. Its interface is based on conversations, so you can easily read an entire discussion without having to click from message to message. Geary is still in early development and has limited features today, but we’re planning to add lightning-fast searching, multiple account support, and much more. Eventually we’d like Geary to have an extensible plugin architecture so that developers will be able to add all kinds of nifty features in a modular way.”

As of today (6 days left), they seem as far as ever from reaching their goal (only 30% complete).

While dominating nearly every other part of the software eco-system, Linux still seems to be failing on the desktop.

If all the domains are identical, in terms of file structure – apart from the hostname – then we can use Apache’s mod_vhost_alias module to simplify the configuration. When combined with mod_rewrite and use of some Apache environment variables (e.g. %{HTTP_HOST}), your configuration can become much more manageable.

So, you can go from multiple virtual hosts configured similar to the below – where all that’s changing between site(s) is effectively the hostname – :

<Virtualhost *:80>
    ServerName mydomain.com
    ServerAlias www.mydomain.com
    DocumentRoot /var/www/vhosts/mydomain.com/public/
    <Location '/'>
         Allow from ALL
         AllowOverRide ALL
    </Location>

    CustomLog /var/log/apache2/mydomain.com-access.log combined
    ErrorLog /var/log/apache2/error.log
</VirtualHost>

To :

<VirtualHost *:80>
    UseCanonicalName Off
    VirtualDocumentRoot /var/www/vhosts/%0/public/
    <Location '/'>
        Allow from ALL
        AllowOverRide ALL
    </Location>
    CustomLog /var/log/apache2/%{HTTP_HOST}-access.log combined
    ErrorLog /var/log/apache2/error.log
</VirtualHost>

Advantages:

1. Once the directory structure has been created (and DNS configured) sites can be served immediately
2. No need to restart Apache,
3. No need to create a new configuration file (one per site etc)

Disadvantages:

1. Perhaps difficult to handle e.g. www.mydomain.com and mydomain.com being hosted from the same directory (leading to symlinks in e.g. /var/www/vhosts).
2. Have to rely on .htaccess files for per-site configuration (which may not be desirable ).

In some circumstances, a library or component may provide a mock or test backend (for example, with the Zend_Http_Client, a test adapter can be plugged in which allows you to pre-set a given response, like in example 1 below). In some circumstances, however, you may not have access to a test adapter, or it may not be possible to inject dependencies into the necessary objects.

The below article shows how you can use hard-coded mock objects (Zend_Http_Client_Adapter_Test), or allow PHPUnit to dynamically create one for you using its test double functionality.

The example code below is based around sending push notifications to the Android cloud messaging platform – for android devices. A service (not included in full) has already been written (Mobile_AndroidPushMessage) which uses Zend_Http_Client internally to send the push message. Its API is similar to the below :

class Mobile_AndroidPushMessage {
     public function addTo($recipientKey);
     public function setPayload($data);
     public function setHttpClient(Zend_Http_Client $client);
     public function send(); // returns boolean
     public function getErrors(); // return array or null if no errors
}

Using a using Zend_Http_Client_Adapter_Test

The Zend_Http_Client library allows you to inject in a different backend adapter – so rather than making an actual request to Google, we can use a pre-determined one. This ensures our test is repeatable and predictable – and fast.

So, firstly, let’s setup a contrived PHPUnit test with a ‘fake’ success message ….

// .... class spec etc.
public function setUp() { 
    $m = new Mobile_AndroidPushMessage();
    $httpClient = new Zend_Http_Client();
    $successAdapter = new Zend_Http_Client_Adapter_Test();
    $successAdapter->setResponse("HTTP/1.1 200 OK \r\nContent-Type: application/json\r\n\r\n" .
       '{"multicast_id":579055555555471,"success":1,"failure":0,"canonical_ids":0,"results":[{"message_id":"0:13612757325555555555d6f9fd7ecd"}]}');
    $httpClient->setAdapter($successAdapter);
    $m->setHttpClient($httpClient);
    $this->androidGcmService = $m; // store it for later use by tests
}
  ...

And test is as follows :

public function testBasicBehaviour() {
    // ... setup payload/recipient key data, can be anything in this example. 
    $m = $this->androidGcmService;
    $m->setPayload('message');
    $m->addTo('someone');
    $ok = $m->send();
    $this->assertTrue($ok, "always returns true, as setUp contains a hard coded success message!");
}

And, as we control the response, we can easily change the above to mimic a failure at the GCM end, using :

public function testErrorHandling() { 
    $m = $this->androidGcmService;
    $adapter = $m->getHttpClient()->getAdapter();
    $adapter->setResponse('HTTP/1.1 200 OKr\nContent-Type: application/json\r\n\r\n I am not json, so something blows up here');
    $m->setPayload('whatever');
    $m->addto('someone');
    $ok = $m->send();
    $this->assertFalse($ok);
    $errors = $m->getErrors();
    // Check contents of the 'errors' array.
}

So – using the Test adapter we can fake error conditions from Google and test our code in isolation.

Using a mock object / test double around Zend_Http_Client

Sometimes we may not always have a test adapter to pass in, or we may instead want to test that (for example) a single HTTP Post request is made. Often a library will not be structured in a manner which lends itself for unit tests, and you may often find yourself needing to sub-class the library and modify parts of its behaviour – however this is probably error prone and time consuming. Thankfully, PHPUnit can help – with its test double support (see here for PHPUnit’s docs on this)

A basic example follows -

public function testPostOnce() {
    $m = $this->androidGcmService;
    // create the doppleganger Zend_Http_Client object, only mocking the 'request' method
    // other methods will function as expected.
    $mock = $this->getMock('Zend_Http_Client'), array('request'));
    $http_body_text = 'some.json.goes.here';
    $fakeResponse = new Zend_Http_Response('200', array(), $http_body_text);
    // tell PHPUnit that the 'request' method should only be called once, and must be a HTTP POST.
    $mock->expects($this->once())->method('request')->with($this->equalsTo('POST'))->will($this->returnValue($fakeResponse));
    $m->setHttpClient($mock);
    // set recipients & payload etc.
    $ok = $m->send();
    $this->assertTrue($ok);
}

When we call the ‘getMock’ function we provide the name of the class we which to ‘extend’. By default, PHPUnit will mock all methods of the specified class – and calling them will return null. In our case we only want to mock one method – ‘request’, and leave all other methods unchanged.

So, the test checks that our class will call the request method once (and once only) and, when doing so, there will be one argument (‘POST’). When the method is called, it will return a pre-populated ‘fake’ response ($fakeResponse).

PHPUnit is doing some magic behind the scenes when creating the mock/double instance, in that it is extending the parent (Zend_Http_Client). This is good – it allows our type hints to function correctly (setHttpClient expects a class of type Zend_Http_Client) and redefines the request() method.

The above is much easier than us creating a new class extending Zend_Http_Client just for the purposes of this test.

As an example, if the Mobile_AndroidPushMessage class actually made a GET request by ‘mistake’ you’d see output like :

Expectation failed for method name is equal to when invoked 1 time(s)
Parameter 0 for invocation Zend_Http_Client::request('POST') does not match expected value.
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'GET'
+'POST'

Using the above, we can ensure that the Http Client is called once, check the parameters used and specify the return value. Again, this is without depending on the actual Google service.

One approach to doing this is to update a database each time a page of interest is accessed, however this isn’t very good for performance reasons. The article below illustrates the problem and how it could be done using an asynchronous background task.

Initial implementation

Starting with code like :

$todaysDate = date('Ymd');
$db->query("INSERT INTO site_hits (date, count)
    VALUES (?, ?) ON DUPLICATE KEY UPDATE count = count + 1", array($todaysDate, 1));

We could use the MySQL ‘ON DUPLICATE KEY’ check, and having a unique key constraint on the ‘date’ field we can ensure we have an incrementing number for each day. This number is being updated in real time (hopefully good). The above can be enhanced slightly by changing it to be ‘INSERT DELAYED’, which should stop MySQL waiting for the query to execute before returning control back to your PHP code.

The problem

The drawback to the above is that every page hit now results in a write to the database – so although your users will see that figure incremented each time they refresh a page, the data will not be cached by MySQL and will result in the server having to read/write from disk all the time. Under normal operations this may not be a problem – but if the site experiences a sudden spike in traffic, you’ll probably wish your I/O bandwidth wasn’t being wasted by the stat counter.

A solution – asynchronous writing to the database

A (hopefully) better approach is to separate out the writing of the counter to the database from that of the page load by using a “global variable” which resides in memory. Periodically this variable is written to disk by a scheduled task/cron job. Pseudo code to provide an example of this is shown below :

On each page load, we instead do the equivalent of :

$counter = memcache_connect('MemcacheServerHost');
$cacheKey = 'my-global-counter';
$count = $counter->get($cacheKey);
if(empty($count)) {
    $count = 0;
}
$count++;
$counter->set($cacheKey, $count);

The above could also be done using APC or XCache variables instead of Memcache, depending on circumstance.

Next, create a cron job which runs every few minutes and does the equivalent of :

$counter = memcache_connect('MemcacheServerHost');
$count = (int) $counter->get($cacheKey);
if($count > 0) {
    $db->query("INSERT INTO site_hits (date, count) VALUES (?, ?) ON DUPLICATE KEY UPDATE count = count + " . $count, array('20130305', $count));
    $counter->set($cacheKey, 0);
}

The above can also be modified slightly again, so we can delay writing to the database if the server load is above a given threshold or we otherwise think the server is too busy. At this point we’d probably want to wrap up our procedural style proof of concept code into a reasonable object – like the following :

// ....
// In each page request :
SiteStatCounter::lazyRecordHit();

Cron job :

// ....
// In the cron job :
if(Server::isNotBusy()) {
    SiteStatCounter::saveHitsToDb();
}

It’s available for free, and provides you with nearly all the functionality of the official web application – but integrates with your Android phone / tablet (e.g. address book sync, photo upload to receipts, maps/directions/navigation and quick dial etc).

It can work in an offline manner – assuming it’s already loaded the data once before, so should work well if you’re using it while mobile. You can even create invoices and so on while offline, and when network connectivity is next established it will submit them.

A few features are available after you purchase an in-app purchase (e.g. Widgets, Address book sync, Invoice creation etc).

Some screenshots follow – we’ve also added a video (which you should be able to see from the Google Play listing; as will be obvious, we’re not video production experts, so it’s probably a bit amateurish - when Kashdroid tops the app charts we’ll hire someone to do a proper version!).

The initial welcome screen

Showing some “pretty” graphs, if you tilt the device over or scroll down along with overview figures

Customer Listing

View your customer’s details – linking into the phone/maps/navigation etc

We’ve created different views / layouts for different orientations – so it’s nice to use on larger devices like tablets.

Invoices

Colour coded to alert you to problematic customers

The app allows you to create/edit invoices,

As well as sending them to customers via email -

The obvious initial caveat is that whatever measures may be detailed within this post are unlikely to be applicable for other scenarios – spammers change tactics.

The site in question runs a WordPress based forum (bbpress) and receives a relatively high level of traffic (due to which, it’s presumably a tempting target for spam postings).

The forum was protected by Akismet, however even with Akismet, an unacceptable level of spam was getting through (Akismet reports that it’s stopped 106,000 spam in the last few weeks).

Initially we :

1. Monitored the Apache access log – from a quick rummage it was easy to see the main pages involved were bb-post.php and bb-login.php (these were the main ones receiving POST requests), often from the same IP address in quick succession.
2. We edited bb-post.php and bb-login.php and dumped the contents of the PHP GET and POST data to disk – so we could see what sort of requests they were receiving.

Based on the above, it was obvious that :

1. A large volume of spammy POST requests originated in Asia, from a relatively small set of IP addresses.
2. A proportion of the spammy comments being posted were in Chinese (yet the site targets English speakers)
3. A proportion of the spammy comments were very large (i.e. >4kb) while most legitimate posts were relatively small (2-4 lines of text).
4. Spammy comments often contained a high number of URLs

While initially it was tempting to go for the quick ‘kill’ and block subnet’s of users using an iptables rule, this wasn’t ideal as it could block legitimate users. The idea of having to maintain this pool of IP addresses wasn’t something to look forward to either – we’d need some way of identifying disreputable IP addresses/clients and automatically blocking them (perhaps with fail2ban in the future).

From experience with tools like SpamAssassin in the past, it seemed most sensible to adopt a scoring strategy with submitted posts – so, for example, we’ve added rules like the following :

1. If the user agent matches a known pattern (E.g. IceWeasel on Debian), spam count += 1
2. If the user’s IP is located in Asia (through use of a geoip lookup), spam count += 1
3. If there are more than 3 URLs within the post content, spam score += 1
4. If the user’s IP is listed in real time blacklists, spam score += 1

Given enough rules, and a reasonable threshold, the chance of wrongly identifying/tagging a post decreases. Now, if the post score is above a specific threshold (e.g. 5) we discard the request returning a non-descriptive error message.

We’ve implemented the request processing before the forum code itself runs/loads, and to minimise overhead, we only check HTTP post requests.

Appropriate logging of requests and data has allowed us to tweak the ruleset over the last 24 hours, to the extent that from a 12 hour period, we’ve identified 3501 spam comments, had one spam posting get through by mistake (which led to a rule update, and was caught by Akismet anyway) and so far, no false positives.

We’ve no intention of replacing Akismet – as it does an excellent job – but certain requests seemed to be easy to target – and by blocking them we can make a significant difference to the site’s performance and users.

The application is used by engineers (so reports a listing of jobs available to them) and allows them to retrieve appropriate details and update job statuses as necessary.

Some interesting features :

• App functionality is customised based on a user’s permissions
• App can run in an ‘offline’ mode – queuing up packets to be sent when network connectivity is re-established (this allows it to work in areas of no network coverage, without affecting the user experience).

Contents:

• Why PhoneGap?
• Disadvantages / Advantages of PhoneGap?
• Disadvantages / Advantages of Native development
• Example(s)

1. Released Android and iPhone apps (Food Hygiene – for ScoresOnTheDoors.org.uk). The Android release is now all native (so quite responsive and fast); both have new graphics and an improved method of searching. (See the iTunes or Play stores). (PhoneGap, Java, GeoSpatial searching).  We’re working on improvements to the iOS variant (better iPhone 5 support), and monitoring the feedback for each.
2. Integrated an existing LAMP application with Investec - allowing the user’s to submit lease / funding proposals (SOAP, XML. PHP5)
3. Busy refactoring an online shop (to improve the mobile experience and enhance administration) (PHP5, MySQL)
4. Updates to the ScoresOnTheDoors website (improve performance, improve business feedback processing, improve data quality, improve the search experience)
5. Refactoring various applications to support PHP 5.4 which we’ve now moved to internally.
6. Migrated two Linux servers (either updating to Debian Squeeze, or to new hardware – for example for the Norton Owners Club )
7. Deployed some performance improvements/enhancements for WordPress based sites we support which use Varnish (E.g. Anorak.co.uk). The cache-hit ratio is now about 90-95% – which is greatly helping responsiveness and server load. (WordPress, Varnish, Linux, MySQL)