tldr: Add this line in the head part of your html-page to do mobile webpages correctly:

<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">

Some facts I have learned the hard way:

• If you do not set the viewport meta tag, iphone will report 980px width.
• If you set width=600px, the devices will report just that: 600px.
• If you set width=device-width, initial-scale= 0.5, the device will alter its reported with accordingly. So an iphone will say 640px, not 320px. (Yes, they report 320px even though the actual pixels is 640). An ipad will say 1536px, not 768px.
• parameters are separated by comma (‘,’) not semicolon (‘;’), although most devices seem to accept both.

I’ve created a page for checking the values reported by CSS3 media selectors. You can also check out how the values are reported if you use initial-scale=0.5, or width=640.

Width reported on an iPad with initial-scale=0.5

Remember to check these out on a mobile device, as the viewport tag is ignored on desktop browsers.

I’m hosting tagdef.com in the Amazon Web Services US-EAST Region, which is located in Virginia. Since this region was partly down during a big storm this summer, I decided to move to Oregon before “Sandy” hit the east coast. In addition to avoiding potential downtime, this was something I wanted to test. How hard is it to move the whole site to another region? Without downtime.

The switch

The first step was to boot a new Ubuntu server and Mysql RDS server in Oregon. When I got the server-instances up and running in Oregon with an outdated copy of the database, it was time to do the switch. It went something like this:

• Set tagdef.com in “read only” mode in Virginia.
• Dump the database in Virginia to a file.
• Import the file in Oregon.
• Verify that the Oregon location (still) worked as expected.
• Point the DNS entries to Oregon.

It can take some time before changes in dns propagate to all users, so people might hit the Virginia server some time after I switched the DNS-entry to Oreon. But since Tagdef is one of those read-intensive applications where many more people read than update, this would not affect most users.

Logging server traffic with Google Analytics

When the storm had passed, it was time to move back to Virginia. I have purchased reserved instances there, so it makes sense to continue to use that region for a while.

Since I really did not know how long the DNS-propagation took, I started logging the server-instance to Google Analytics using Non-Interaction Events.

Moving back from Oregon

By setting up custom segments in Google Analytics, it is easy to see when the switch happened, and that all the traffic was moved within an hour or two.

It’s also interesting to see the response time measured during the Oregon-switch. Since many of the servers that is used to measure uptime is located in Europe, the latency was increased when the servers were moved to the west-coast:

Cost

Since I had to set up the web-server from scratch, it cost me about three hours of labor. There are methods of moving a machine-image between regions, but it didn’t work for me.

The cost of the web-server, load balancer (for SSL termination) and database-instance running for two days: 12 dollars.

In June Google announced that Google Website Optimizer will be replaced by Google Analytics Content Experiments. With the new tool it seems hard to do a/b testing of dynamic websites, at least I haven’t figured out how to do it.

I use custom events in Google Analytics on Tagdef for various measurements, including the time spent by the server to generate the page. This event is a so called non-interaction-event, and is fired for every page.

Since you can create custom segments based on events, I decided to make my own poor man’s a/b-test:

1. For each new visitor, assign by random if this should be the original page (a), or the modified version you want to test (b).
2. Store this decision in a cookie, so the user gets the same experience on repeat visits.
3. If b, Do the appropriate modifications to the page with Javascript.
4. Send a custom event indicating if this is an a or b to Google Analytics.

The whole script can be like this:

var A_SEGMENTNAME = "OriginalPage";
var B_SEGMENTNAME = "BlueSignUpButton";
var abCookieName = "my_abtest";
var abTestSegment = $.cookie(abCookieName);
if(abTestSegment === null) {
    abTestSegment = A_SEGMENTNAME;
        if(Math.random() >= 0.5)
    abTestSegment = B_SEGMENTNAME;
    $.cookie(abCookieName, abtestSegment, { path: '/', expires: 30 });
}
if ( abTestSegment === B_SEGMENTNAME) {
 //Do something to modify your page
}

$(function() {
    _gaq.push(['_trackEvent', 'usersegment', abTestSegment, '',1, true]);
});

This code uses jQuery and the jQuery Cookie Plugin.

Custom segments

In order to compare the two groups of visitors, I created custom segments in Google Analytics.

Click Advanced Segments near the top of the page, and then New Custom Segment.

Remember to add segments for both the original and the variant.

Now I can explore the two segments in Google Analytics, and see if there are significant changes in metrics like pageviews/visit, and Adsense revenue.

Does this change make people use my site more, and is this different for mobile users?

Does this change make me more money?

If you have set up Goals in Analytics, you can compare the conversion rate for the two groups like more traditional a/b testing does:

Multivariate, and no overhead

This seems to me to be more flexible than the old Google Website Optimizer, as you get all the Analytics info for both segments. And you can add variations easily by extending the javascript and create more custom segments, which I think Content Experiments does not support. Also, it does not require any additional javascript downloads if your site is already using Google Analytics.

When changing an ASP.NET MVC3 application to run in integrated mode in IIS 7.5, we ran into some problems with global error handling. We wanted to catch all unhandled exception in order to log the error, and we wanted to output a custom error message.

In order to catch all unhandled exceptions, we chose to (still) use the Application_Error callback in global.asax.cs, and remove the default behavior:

public static void RegisterGlobalFilters(GlobalFilterCollection filters) {
    filters.Add(new HandleErrorAttribute()); //Remove this line
}

When removing that filter Application_Error gets called like we want it to.  We then discovered that no matter what we outputed, we got the same old IIS “Internal server error” page.

Layers, like an onion.

It turns out there are two layers of error-handling here. First we do our job in .NET, but then IIS figures out something is wrong (since we returned HTTP 500), and slaps it’s own content in the response. After digging around, I found the answer in a blogpost: Put this line in your Application_Error method:

    Response.TrySkipIisCustomErrors = true; 

I would like to meet the guy (or girl) who came up with that name. It’s descriptive, and it seems quite honest. Set this one to true, and we’ll do our best to skip that horrible IIS error handling for you…

Despite its modest name, it worked perfectly. Now we got to log our errors, and we got to display our own content.

Content-type

This last tweak has little to do with error handling in MVC, but I thought I should mention it in case anyone else experience the same behavior. When outputting our custom html, it suddenly got rendered as plain text, not interpreted as html. It turned out the content-type header was not set. To do this, set the correct content-type in Application_Error:

   Response.ContentType = "text/html";

Earlier I wrote about using Google Analytics to log the time spent generating content on Tagdef.com. One day I looked at the stats, I realized that this time had doubled:

The observant reader might point out that the time spent server-side increased by only 15 milliseconds, and that no user would ever notice this. I know, you’re right. This particular performance-hit is no big deal by itself. But if small incremental changes like this are introduced regularly without any benchmarking, it might be hard to fix the code when you discover that the server spends half a second serving each request.

Finding the culprit

The source control history confirmed that I had made several changes in the period where the performance suffered, but I couldn’t find anything that should cause this when reviewing the code (yes, I often compensate my bad memory by reading my own commits). Since this gave me no clues, I started profiling the code.

The profiler gave me the answer: I had introduced code that should write to the database in rare circumstances, but a bug caused this code to be executed on each page-request. Even though the database-write failed (so nothing showed up in the database), this doubled the total time spent serverside.

Once I discovered the bug, fixing it was trivial and resulted in 20 characters added code.

This was clearly a bug. The code I wrote did not behave as I intended it to,and it impacted the performance. But since it did not cause any malfunction for the user (or owner), it could easily have been left there without anyone noticing. This makes me wonder how many other bugs like that there might be out there, undetected.

Perhaps a good unit-test could have caught this one. But adding statistics as another layer of quality control proved to be useful, and made this error stand out once I glanced at the chart. It also demonstrated that the additional functionality did not slow the site down. Once the bug was fixed, the code went back to its earlier performance.

-MrCalzone

In my last (and first) post I wrote about server side optimization at Tagdef.com. Tagdef is a dictionary for hashtags, with user generated content.

This post looks at some of the things I’ve done to make the page load faster in the browser, once the bits are transferred over the wire.

Most of the client side optimization done at Tagdef.com are based the Best Practices for Speeding Up Your Web Site from Yahoo. I will not try to write a comprehensive guide like that, but rather focus on some details I’ve learned while implementing this on Tagdef.

Content Delivery Network

Most of the static files on tagdef.com are hosted on Amazon Cloudfront. Actually, the only static file hosted on tagdef.com is the stylesheet. As the css-file is the first file to be loaded after the actual html-page, I figured it will be faster if the browser doesn’t have to do another dns-lookup to load the stylesheet.

Amazon Cloudfront is a Content Delivery Network, with servers (edge-locations) all around the world. Using a CDN has at least two advantages:

• The files are delivered from a location close to the user, making the download faster.
• Since these files are no longer handled by your web-server, it gets more available resources to what it is supposed to do (e.g. generate your dynamic content), and you will be able to handle more traffic.

Compressing files on CloudFront

Text-files like css and javascript can be compressed before sent from the server, saving typically 70% of the total size. This is supported by most browsers and servers, so why do I even mention it?

Amazon Cloudfront doesn’t support serving compressed content. That is, there is no button or setting where you can say”compress this file”. But the solution is rather simple: Just gzip the files on your server, and add the header Content-Encoding:gzip to the file when uploading it to CloudFront (Actually you upload it to S3, and distribute it via CloudFront).

Setting HTTP headers on CloudFront

This means that all browsers will get the compressed files, even if they don’t support it. The most “modern” browser that doesn’t handle this seems to be Internet Explorer 6.0, if you haven’t updated it in ten years! If your browser can’t handle gzip, it probably can’t handle the content inside the file either. I’d rather spend my time enhancing the experience for 99% of the users, than spend time making it work for someone running Netscape 4.

Since you pay for each byte transferred from CloudFront, it is also good for the wallet to compress and minify files.

Cache files forever, change the name when updating them

As you can see in the screenshot above, I’ve specified an Expires date far in the future. This means that the browser will only fetch it once, and just read the local copy when it needs it next time. Instead of trying to control when the file is refreshed, I simply change the name of the file whenever I update the content. Right now the css file for Tagdef.com is called style_84_min.css. The next time I make changes to the stylesheet, I call it style_85_min.css, and update the reference in the html-document. No user will have this file in their cache, so all users will get the new stylesheet when visiting the site.

Specify sizes

In addition to specifying image-sizes, I’ve found that specifying width and height on other elements helps the page not “jump around” when loading. If a box near the top of the page changes size during rendering (maybe because some element inside it is loaded), the whole page has to be re-arranged by the browser. I think the site is experienced as smoother, faster, if the top-part (or other fixed sections) of the page does not move at all when navigating it. It feels a bit more like a native application…

Tools

• Google page speed and Yahoo YSlow. Both these are plugins to your browser that check you page against best practices. They indicate what you do right and what you can improve, and give you a score.
• Pingdom tools is an online tool, that in addition to checking the structure of the page, also measures the actual speed of loading the page.
• Google Analytics Site Speed. The speed you measure in your own browser, or on the fast servers from Pingdom, might be very different from what the actual users of your site experience. Last year Google Analytics started measuring the time spent rendering the page in the users browser. Since this can be combined with the usual filters in Analytics, you can get a lot of information from this. Like how fast your site is on mobile devices, or for people in Canada using Chrome
  

  Site speed for Canadians using Google Chrome (I've edited the screenshot to remove the number of visitors, and making it narrower to fit this page).

  This shows that the time spent getting the files from the server is only a small fraction of the total time spent loading the page in the browser, even after these optimizations.

  The most effective speed improvements can probably be done by optimizing the client side of your site. Running these tools and fixing the most severe issues can be big wins. But keep in mind that the end result is a combination of the time spent on the server as well as in the browser. If the server had spent one second generating the files in the above example, the site would have been 50% slower for a lot of people, with a total of three seconds instead of two.

  -Mrcalzone

Tagdef.com is a website for looking up and defining hashtags. Think of it as Urban Dictionary for hashtags.

Speed is a feature

When designing and building Tagdef.com, speed has been one of the key focuses. A faster website makes the users happy, and a website that can handle a decent peak in traffic makes the owner sleep well at night.

Both Google and Amazon have done studies which indicate that the user-experience is directly linked to the speed of the page:

After a bit of looking, Marissa explained that they found an uncontrolled variable. The page with 10 results took .4 seconds to generate. The page with 30 results took .9 seconds.

Half a second delay caused a 20% drop in traffic. Half a second delay killed user satisfaction.

Even though studies show that most of the time is spent rendering the page in the users browser, a snappy server helps. And if the server spends less time processing a request, it can process more requests per second on the same hardware (all things equal).

Varnish

When an item is requested from the web-server, it is first handled by Varnish. Varnish is a free, high performance, http cache server. If the requested page exists in the Varnish cache, Varnish will return it to the users browser without any communication with Apache. Varnish is configured to never expire css, javascript and images. So Apache is only bothered with generating the dynamic content, the actual web-page.

There are several elements on a typical tagdef-page that goes stale quickly:

• The relative time of the definition (“added 13 minutes ago”)
• The list of recent tweets for this hashtag
• Related tags
• The actual content of the definitions.

Because of this, I’ve chosen to have a short time-to-live for dynamic pages in Varnish, typically 10-20 minutes. As a result, the long tail of seldom visited pages will expire from the Varnish cache. But that’s OK. The main purpose of Varnish in this setup is to cache the most popular pages. If the same page is hit many times in a short time-period, most of the requests will be taken care of by Varnish. This is a typical scenario when a hashtag is trending, or every Friday when people search for #ff.

Memcache

Let’s say the requested definition-page was not in the Varnish cache. Varnish asks Apache for it, and good old php-code is executed. When the php-code needs some dynamic data (which means most of the content on Tagdef), it first asks a memcache-server for it. A request to MySQL is only done if the content is not found in Memcache. Some of the content in the memcache-server expires automatically, other content is only thrown out if the content is modified.

Invalidation

If a user adds a definition, she expects to see that definition when returning to the page. This would not work if Varnish was simply returning the old page from memory. So when a user adds a definition, Varnish (and Memcache) is told to throw out the old content for that page.

Refresh the cache asynchronously

Most of the MySQL-requests are fast (less than 50ms), and the page should still feel pretty fast even if no cache is used. But for some content, the requests take more time. Content like this is requested and updated in the cache regularly by background processes, so the users should always hit the cache and get a fast response.

External resouces

Tagdef uses data from Twitter to show recent tweets and trends, and Google Translate to translate any non-English definitions. All communication with these external APIs is done asynchronously. This means that Tagdef will work even if Twitter is down, but some features will be missing. The data is cached in Memcache, so we don’t abuse the external APIs by asking over and over for the same information. This also makes the external content load instantly, if it is found in the cache. If not, it is loaded asynchronously in the browser (Ajax).

Logging

Statistics are fun. But if I were to log the performance of every request, that might itself have messed up the performance. And besides, Google is much better at presenting numbers than I am. So I use Google Analytics to track time spent generating the page.

Time spent generating the front page, in milliseconds.

Profiling

I’ve spend some time profiling the PHP code, and removed some bottlenecks. Among other things, I discovered that I spent a significant amount of time connecting to the MySQL server, even though I didn’t need MySQL in every request. I made the code a bit lazy, so it only connects to MySQL when it needs to. As a bonus, the most frequently used parts of the website will work as normal, even if the MySQL server is down for a couple of minutes.

Evil

It is popular to say that premature optimization is the root of all evil. Most of the optimization done based on the profiling might be well into premature optimization land. And could I have done without Memcache and Varnish? Probably. Certainly for 99% of the time. But the page would have been a bit slower for everyone. And Google indexing tens of thousands of pages a day might have caused a problem. Not to mention the horror of realizing that your site crashed when it finally got some major publicity.

And besides, it’s fun to see the front-page generated in 3.2 milliseconds (and fully load in 600 milliseconds). I’m an engineer, because I can is a valid reason for me. And when that makes the user-experience better, the server-bill smaller, and the site stay online, all the better.

-MrCalzone