Use Netbeans.

If I could offer you, one piece of advice for the future – NetBeans would be it.

The long term benefits of NetBeans has been proved by Rockstar developers, whereas the rest of my advice has no basis or reliable than my own meandering experience. I will dispense this advice, now.

Enjoy the power and control of your IDE. Oh, never, you won’t understand the power and control of your IDE until you use Dreamweaver, but trust me within 20 minutes, you’ll look back at your Dreamweaver projects and recall in a way you can’t quite grasp now how horrendous that application really is. You are not as nooby as you imagine.

Don’t worry about scalability, or worry, but know that worrying about scalability is as effective as trying to solve a fizz buzz problem by forgetting modulus.

The real troubles in your life are apt to be things that never crossed your worried mind, the kind that blindsides you at 8am on a Monday morning during some unannounced Scrum meeting.

Code one thing every day that is Open Source.

Refactor.

Don’t be reckless with other people’s code; don’t put up with people who are reckless with yours.

DRY.

Don’t waste your time on Internet Explorer. Sometimes you’re in scope, sometimes you’re not. Race conditions are a bitch, and in the end, it’s normally just that trailing comma.

Backup your MySQL databases; delete your /dev/null logs. (if you succeed in doing this, tell me how).

Keep the first application you coded; delete and uninstall Visual Basic.

Namespace.

Don’t feel guilty if you don’t know if you want to use Dojo or jQuery. The most interesting startups that I know didn’t know during a $1M seed round what their product even was; some of the most interesting $1B valued companies still don’t.

Get plenty of TechCrunch. Be kind to your fingers — you’ll miss them when they’re gone.

Maybe it’ll compile, maybe it won’t. Maybe you’ll have followers, maybe you won’t. Maybe you’ll sell at $800M; maybe you’ll prove Euler’s theorem was wrong and Pi is actually 42.

Whatever you do, don’t commit your code too much, or rebase either. Your choices are bug fixes, patches, or new features – so is everybody else’s.

Enjoy you API docs, use them every way you can. Don’t be afraid of commenting code or what other people think of it; correct indentation is four spaces, always.

Dance… even if you have no where to do it but your own bedroom.

Read the fucking manual (even if you don’t follow it).

Do no read w3schools; it will only make you feel clever.

Get to know your IRC friends; you never know when they’ll be gone for good.

Format your StackOverflow answers: it’s an easy way to get upvotes and the most likely way to be an accepted answer.

Understand that contributors come and go, but that a certain few you should hold on to. Work hard to learn new technologies and web standards, because the older you get, the more you realise that knew less today than you did when you were young.

Live in Palo Alto once, but leave before it makes you hard.

Work at Apple once, but leave before it makes you soft.

Google.

Accept certain inalienable truths, advertising costs will rise, Facebook will change their API and Microsoft will release an IE 13; and when they do, you’ll fantasize that when Mosaic was around, hyperlinks were new, stylesheets didn’t exist, and the back button was a game changer.

Viva la Ubuntu.

Don’t expect anyone else to debug for you. Maybe you will have a sympathetic colleague, or Unit Test everything, but you never know when either might run out.

Don’t mess too much with your code, or by the time you are in production, it will be unintelligible.
Be careful of eLancers, but be patient with new employees. Perfect interviewing techniques; They are a way of recruiting new talent, discarding of the old, and not paying too much for what they are really worth.

But trust me, on the NetBeans.

About a year ago I researched the easiest way to throttle some of my resource intensive requests. These were also the requests where the user might accidentally hit the “Send” button several times, and where I definitely only want it to be sent once. Anyway, on this current project, I wanted to do it again and couldn’t remember for the life of me how to throttle requests so that the request will only happen once in a set period of time. So I’ve dug through my old projects and managed to dig the code out.

I can’t actually remember if I wrote this myself, it’s been so long, so if anyone happens to stumble accross something similar from pre-1 year ago, I will post a link.

Turn’s out I couldn’t find it anywhere so I guess I was the author. Lots of similar usecases though.

Version 1

Version 1 (This will fire off the function only once and only at the end of the delay period. This means that you have to wait at least until the end of the delay period).

dojo.declare("foo", null, {
    constructor : 	function(  ){
        // Make the connection so that clicking the
        //button will call the throttle function.
        dojo.connect( dijit.byId("button"), "onClick", this, "throttle"  );

    },
    // The period that you want to throttle for.
    //750 = Only request it once every 750ms.
    delay : 750,
    throttle : function(){

        return dojo.hitch(this, function() {
            if (this.executionTimer) {
                clearTimeout(this.executionTimer);
            }
            // The throttling. The
            this.executionTimer = setTimeout(dojo.hitch(this,function() {
                console.log("Throttling complete");
                this.loadPage();
            }), this.delay);
        })();
    },
    loadPage : function(index){
    // The Xhr for loading some external page.
    }
});

Version 2

Version 2 (This will fire off the method once immediately, and wait the delay before ever firing it again).

dojo.declare("foo", null, {
    constructor : 	function(  ){
        // Make the connection so that clicking the
        //button will call the throttle function.
        dojo.connect( dijit.byId("button"), "onClick", this, "throttle"  );

    },
    // The period that you want to throttle for.
    //750 = Only request it once every 750ms.
    delay : 750,
    throttle: function( id ) {
        return dojo.hitch(this, function() {
            if (this.executionTimer) {

            } else {
                // First time the Save has been called.
                this.loadPage );
                this.executionTimer = setTimeout(
                        dojo.hitch( this, function() {
                                  delete this.executionTimer; }
                         ), this.delay);
            }
        })();
    },
    loadPage : function(index){
    // The Xhr for loading some external page.
    }
});

Version 3

Version 3 (This will fire off the method once immediately, and then throttle any requests that fall within the delay period so that you don’t choke up requests).

return dojo.hitch(this, function() {
            var firstThrottle = false;
            if (typeof this.executionTimer == "number") {
                // Reset the Timeout if it is a number (Meaning that we are currently in the process of throttling.
                clearTimeout(this.executionTimer);
            } else {
                firstThrottle = true;
                this._loadFile(e);
                this.executionTimer = setTimeout( dojo.hitch( this, function(){
                    clearTimeout(this.executionTimer);
                    delete this.executionTimer;
                }), this.delay);
            }
            // The throttling. The
            if( firstThrottle == false ) {
                this.executionTimer = setTimeout(dojo.hitch(this,function() {
                    delete this.executionTimer;
                    this._loadFile(e);
                }), this.delay);
            }
        })();

I just stumbled on this revelation from Vic Gundotra (+GooglePlus) that Facebook are actively censoring certain posts that users are making. While Vic is diplomatic and doesn’t explicitly say that this is what is happening, it’s clear that this censorship is no mistake.

This is a further step to their developer policy changes last week, where applications that feature on Facebook are not allowed to provide any links to competiting social networks.

“I.11. Apps on Facebook may not integrate, link to, promote, distribute, or redirect to any app on any other competing social platform.

While, I can (somewhat) agree and understand their refusal of allowing advertisements by competing services, which they restrict in their advertising guidelines,

We may refuse adverts at any time for any reason, including our determination that they promote competing products or services, or negatively affect our business or relationship with our users.

I can’t believe that they are now  overtly censoring posts of users, without their knowledge. Major shit storm to follow…

 
(Video created by rss1357.)

Links on Facebook

It worked for Castor Troy and Sean Archer.

Completely different post today, after reading one of my friends comments on Facebook, I decided to throw my thoughts into the mix about what Google+ might actually achieve through their platform.
Originally, I started commenting on his post, then after ten lines, I moved it to a post on my own facebook wall, then after 10 paragraphs, I figured this really needs to be a complete post.

Anyway, his comment which stirred my response.

So his concerns are clear, privacy. Everyone knows that Google constantly try to hit home their “do no evil”/”don’t be evil” tenet, but this still hasn’t prevented them from being the focus of several highly publicised invasion of privacy cases. I’m extremely cautious about what I make public on my social graph, and I would say it is not the consequence of not wanting to share things socially, it’s because I know what happens with the information which I do share.

You will never find my liking anything on Facebook, or posting to any public group, or having anything in my “About” information. In fact, if I was actually able to de-list and remove my relationship I would. Currently, all I see when browsing Facebook are advertisements about dating sites and making money over the summer by donating certain bodily fluids (and it’s not blood). Facebook publishers who advertise, know that I am single, and this makes targeting advertisements towards me incredibly easy.

Now, to link this back to my friends comment, this is pure exploitation of personal information for corporate purposes. I don’t think it matters what advertisement you see, 99.99% of the time, you were better off without seeing it, you receive no added value by seeing the advertisement.  I may be a special case, but have yet to see anything that I want to click on.

I do not mind business intelligence when it adds value to the service which I am subscribed up for. When I visit Amazon, I like that they harvest my buying history and user details, because I receive extra value from their service. I go to Amazon to shop, and they add value to my shopping experience. However, this can’t be said to be the same for Facebook, I go to Facebook for social reasons, but my public domain information is being exploited without any value being added to my social experience, all the contrary, because I know how my information is being used, it has stunted my social experience.

Now, this perspective might be entirely different if I did receive some additional value from Facebook when I shared personal information. What value can they offer me in their current services?… currently, none. However this doesn’t mean that other services can’t benefit from my social graph and personal information. And this is exactly what is happening at the moment with Facebook Connect and their Facebook API, however only to a limited extent. Look at this project which allowed you to scrape your own personal Facebook for your friends emails, to allow you to export it to Gmail, well you can’t because it got shut down by Facebook. Facebook continuously suckle away at data from other entities users, yet refuse to release, where willing, information about users who request it.

Facebook are refusing to let users export their entire social graph, while they continuously importing data from other services at an alarming rate. There is no reciprocity, and a data-wars going on which is seeing Facebook locking down their platform, in a position which makes it impossible to transition from Facebook to another service.

Now, this is where I think Google+ will come in, and some good will hopefully come out of this: a more open social graph, and cross-interoperability between social networking.

Want to take YOUR content, and your social graph in *real time* from Facebook, well you can’t. The  Facebook API, while it does unlock a lot of social connections about a user, however it still doesn’t, and won’t ever (as it stands) allow you to take what you have created and port it to another social site.  Meaning that what you create here can’t be taken with you to any other social site, ie G+, Diaspora..
But, even if Google+ doesn’t become equals with Facebook, hopefully they will become large enough that cross interoperability will become a neccessity. When email first came out, you couldn’t email people outside of your network. After large scale adoption this was obviously totally reworked, and today we have the @ symbol. When mobile phones started out, you couldn’t call someone outside of your network… need to call someone in a different cell tower… well you couldn’t. Now all phone carriers work together.

And this is where I can return back to the whole “added value” argument above. If this information could be easily transacted between social networks, then I am immediately receiving added value to my social experience. I go to Facebook for social, I share everything, and I get everything I share back. In sharing on Facebook, I would be able to receive more targeted search results on Google, which I do want.  Yes, I will also be targetted by advertisements in Google, but the difference between Google targetted advertisements and Facebook, is that Google offer me so much added value when I give them personal information, which Facebook just can’t and won’t.

So, hopefully, if anything good comes from Google+, it is that it changes the perspectives of users who have only known Facebook to deliver them a social experience. Users should want to see cross interoperability, and migrating to Google+ which already allows a full data dump of your information should help remove the wool from over the sheep’s eyes.

This is going to easily turn into a wall of text, TL;DR post, so I have provided a summary of it all at the top. Make of it as you will, I just wanted to see how I could get things to perform by using a standard Drupal Install, with about 10 modules… the usual really:

• - Views
• - Panels
• - CSSTools
• - Context
• - Webform etc

Anyway, I am going to kick this off with the details of the testing environments that I am using for this.

- The server which will be using AB is not in the on the same network as the target machine.

- Both servers are running Ubuntu 10 Maverick AMI code : ami-e59ca991

- Target machine running the webserver is running PHP 5.3 and Apache 2.2.x

- All benchmarks are calling a page which requests a page that uses Views. (Which means I know that I will be getting at least 30 MySQL IO for just this page).

Step 1 - No caching, no varnish, no pagecache, no APC, KeepAlive Off, The bog standard install.

ab -k -n 100 -c 100 -g step1.txt http://aptenex.com/how-it-works

…. Okay that didn’t go well. My remote server just crashed… Time to reboot and try it again with top running so I can see where I’m being killed. d

Okay so I managed to crash it again by reducing the concurrent requests down to only 50. I didn’t manage to get the screenshot when it was maxed out at top, but this was the screen at crash.

What can we tell from this? Memory is a massive bottleneck on the system. Which isn’t suprising when you can’t run Drupal with 128MB of memory_usage allowed per PHP request.

Test 1 (again) – No caching, no varnish, no pagecache, no APC, KeepAlive Off, The bog standard install.

ab -k -n 100 -c 2 -g step1.txt http://aptenex.com/how-it-works

Well it didn’t crash this second time. Here is the ab output, with 100 requests, at 2 concurrency.

Very pitiful…

Requests per second: 0.51 [#/sec] (mean)
Time per request: 3957.105 [ms] (mean)

Benchmarking aptenex.com (be patient).....done

Server Software:        Apache/2.2.17
Server Hostname:        aptenex.com
Server Port:            80

Document Path:          /how-it-works
Document Length:        20963 bytes

Concurrency Level:      2
Time taken for tests:   197.855 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      2138900 bytes
HTML transferred:       2096300 bytes
Requests per second:    0.51 [#/sec] (mean)
Time per request:       3957.105 [ms] (mean)
Time per request:       1978.553 [ms] (mean, across all concurrent requests)
Transfer rate:          10.56 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       93  127 153.2    100    1479
Processing:   975 3823 8307.6   1157   36766
Waiting:      679 3379 7710.9    886   33240
Total:       1073 3950 8380.0   1257   36870

Percentage of the requests served within a certain time (ms)
  50%   1257
  66%   1308
  75%   1332
  80%   1405
  90%  14978
  95%  30174
  98%  34423
  99%  36870
 100%  36870 (longest request)

Second Test…
Repeat Test 1.

Changes made.. Simple…

ab -k -n 100 -c 2 -g step2-k.txt http://aptenex.com/how-it-works

The results are pretty similar.

Concurrency Level:      2
Time taken for tests:   207.347 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      2138900 bytes
HTML transferred:       2096300 bytes
Requests per second:    0.48 [#/sec] (mean)
Time per request:       4146.935 [ms] (mean)
Time per request:       2073.467 [ms] (mean, across all concurrent requests)
Transfer rate:          10.07 [Kbytes/sec] received

Test 2. Enable APC

This is where I expect to get at least a 50% performance improvement. Not sure what to expect, but I would at least like to see the time per request decrease at least 2 fold.

ab -k -n 100 -c 2 -g test2-c2-k.txt http://aptenex.com/how-it-works

First Test

Concurrency Level:      2
Time taken for tests:   49.233 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      2138900 bytes
HTML transferred:       2096300 bytes
Requests per second:    2.03 [#/sec] (mean)
Time per request:       984.656 [ms] (mean)
Time per request:       492.328 [ms] (mean, across all concurrent requests)
Transfer rate:          42.43 [Kbytes/sec] received

Second Test

Concurrency Level:      2
Time taken for tests:   87.270 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      2138900 bytes
HTML transferred:       2096300 bytes
Requests per second:    1.15 [#/sec] (mean)
Time per request:       1745.396 [ms] (mean)
Time per request:       872.698 [ms] (mean, across all concurrent requests)
Transfer rate:          23.93 [Kbytes/sec] received

I also feel a little more adventurous and am increasing the concurrent requests up to 5 users.

ab -k -n 100 -c 5 -g test2-c5-k.txt http://aptenex.com/how-it-works

Concurrency Level:      5
Time taken for tests:   65.481 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      2138900 bytes
HTML transferred:       2096300 bytes
Requests per second:    1.53 [#/sec] (mean)
Time per request:       3274.039 [ms] (mean)
Time per request:       654.808 [ms] (mean, across all concurrent requests)
Transfer rate:          31.90 [Kbytes/sec] received

Wow. So this was actually a lot better than I thought. 2 requests per second. Okay, that’s still absolutely dismal.
But the important thing to note is that I was able to request each page in a tenth of the time. Previously it was 4146ms per request, n

ow it is 492.328ms

Test 3. Enable Caching in Drupal Core.

ab -k -n 100 -c 5 -g test2-c5-k.txt http://aptenex.com/how-it-works

Concurrency Level:      2
Time taken for tests:   23.229 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Keep-Alive requests:    0
Total transferred:      1923002 bytes
HTML transferred:       1880900 bytes
Requests per second:    4.30 [#/sec] (mean)
Time per request:       464.580 [ms] (mean)
Time per request:       232.290 [ms] (mean, across all concurrent requests)
Transfer rate:          80.84 [Kbytes/sec] received

Another 50% performance Increase. Double the amount of requests per second, and a time request per page of only 232ms. (Still about 180ms more than I actually want it to be but we are getting there).

Test 4. Enable Varnish.

The final step, add a reverse proxy cache application into the mix. What do I want to see?.. I actually don’t care, anything *must* be better than 4 requests per second. If I can get it to around 300 requests per second, then I will be pleased. Anything close to 1000 requests I’ll be ecstatic.

Server Software:        Apache/2.2.17
Server Hostname:        aptenex.com
Server Port:            6081

Document Path:          /how-it-works
Document Length:        18514 bytes

Concurrency Level:      300
Time taken for tests:   11.706 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Keep-Alive requests:    10000
Total transferred:      190260000 bytes
HTML transferred:       185140000 bytes
Requests per second:    854.29 [#/sec] (mean)
Time per request:       351.168 [ms] (mean)
Time per request:       1.171 [ms] (mean, across all concurrent requests)
Transfer rate:          15872.83 [Kbytes/sec] received

Overall, pretty impressive. I managed a percentage increase of 167407.84% in the amount of page requests I could handle per second.

• Start: 0.51
• End : 854.29

And additional I reduced the page loading time per request from  1978ms to 1.17ms (concurrently) which is a overall speed gain of … *a lot*.  A speed decrease of 99.94%. Ouch.

I’m about to do a quick little mini series on benchmarking a fresh Drupal site with APC and thought before I start that, lets try and provide a guide on the steps which I take to make it possible.

There is absolutely no reason why you would ever not want to run APC/memcache on your server which servers PHP applications. My Zend Framework sites with 220+ PHP files received a page load boost of a factor of x 1.8 just by enabling APC alone. I wouldn’t be at all suprised if I see similar results with Drupal too.

Anyway, this is a short post and here is the condensed “how to”.

sudo apt-get install php-pear

sudo apt-get install php5-dev

sudo apt-get install apache2-dev

pecl isntall apc

Once you have installed APC, you then need to enable it in any .ini file which will be parsed.  I decided to create my own APC.ini file because I know that I will need to enable several APC options and it just makes life more simple.

Navigating to /etc/php5/apache2/conf.d I then created a file called apc.ini and added on the first line:

extension=apc.so

That is it. Now all you need to do is reload apache.

sudo service apache2 reload

sudo /etc/init.d/apache2 reload

So, you have just created your EC2 instance and it is launching.
During the launch process you will have generated a key, this is a one time download, which you can not recover ever… *ever*.

If you lose this key, you will need to generate a new key for your instance, and then use that.

Anyway, with your key downloaded, you can use PuttyGen to create your public/private key pair.

You can download Putty Key Generator from : http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html

Once you have it downloaded launch it and then load an existing private key.

When you have started the Putty Key Generator application, you then need to click :

Conversions > Import Key.

And import your MyKey.pem file.

The import is instant, then click, “Save the Private Key”.

* You do not need to do anything to do with “Generating Key” * Just important and save private, that is it.

If you want to have a password for your key, you can enter one.

Once you have your .ppk file you can then use that to connect to your EC2 Instance.

- Tip : It is a good idea to save your files which you will use for ssh in your C:/users/%username%/.ssh directory (if you are on windows of course).

See Part 2 of the series on how to do that.

Just a quick post to explain the quickest way to add a virtual host to WAMP.

There are a tonne of ways to do it, but I have recently started generating a tonne of  VirtualHosts (VHosts). I have always just edited the httpd.conf file which you can find in your /conf directory of WAMP. But recently I have started letting WAMP create aliases for me and then edit the alias file in order to move the alias and turn it into a VirtualHost. So to start with, add a new alias.

Step 1. Add an Alias to WAMP.

Click on WAMP then Apache, then Alias Directory and Add an alias.

When the screen pops up enter the domain which you would like to use as a VirtualHost. For instance, www.example.com. (This will create an alias like localhost/www.example.com which obviously isn’t what we want but what it achieves is a nice directory structure and custom conf file which we can edit ourself).

Next enter the public document root for your alias and hit enter again.

Step 2. Edit your Aliases config file.

Once you have edited the alias config file you can add your virtual host information.

Here is what I use.

NameVirtualHost 127.0.0.1
<VirtualHost 127.0.0.1>
ServerName website.com
DocumentRoot "C:/website/public/" 

<Directory "c:/node">
allow from all
order allow,deny
# Enables .htaccess files for this site
AllowOverride All
 </Directory>
</VirtualHost>

That is it. Restart Apache through WAMP and your new website.com will be working as a VHost.

Of course, you can edit your httpd.conf directly to add the VHOST, but I like having seperate config files and this process saves time in the log run. For example, if I wanted to add a vhost specific directive you can add it in your alias file. (Which isn’t really an alias file anymore).