App Engine Series #3: Using the Webapp Framework

Important: As of July 2015, this tutorial no longer works, as App Engine has shut down the Master/Slave Data Store that the application uses. We are keeping it online for reference purposes and you can still download the code, but it needs to be converted to the newer High Availability Data Store to work.

In this third part of our App Engine series, we will start writing the python code that drives our uptime dashboard. We will be using Google's Webapp framework, which is a lightweight tool for building applications on the platform. It also comes bundled with the SDK, so we don't need downloading or setting up a separate framework.

If you are new to the series, the best starting point would be the leading article, which serves as the series' index. You can also read the previous part, where we discussed the directory structure and file organization.

The MVC Pattern

Our dashboard is built using Google's webapp framework, which will help us organize our code in the Model-View-Controller pattern. First, however, lets say a few words about the MVC pattern we are going to be using on App Engine.

As the platform uses a schemaless object datastore, we are not constrained by a fixed table structure to store data. We can directly save an entire Python object and retrieve it later. These objects are our models. They handle reading and writing to the datastore, so that developers can concentrate on other parts of the application.

The controllers are bound to specific URLs and are executed when a request to that URL is made. The views are HTML templates that take care of displaying the application data according to a set of template tags (more on these in a moment).

To learn more about the MVC pattern and its flavors, start by reading this Wikipedia entry.

The Models

For your application to be able to store objects to the datastore, they need to extend the db.model class, exposed by App Engine's datastore API. You can add fields to these objects from one of the supported datatypes.

Here is what the models used in the dashboard application look like:

models/models.py

#!/usr/bin/env python

from google.appengine.ext import db

# These classes define the data objects
# that you will be able to store in
# AppEngine's data store.

class Ping(db.Model):
    responseTime = db.IntegerProperty()
    date = db.DateTimeProperty(auto_now_add=True)

class Day(db.Model):
    averageResponseTime = db.IntegerProperty()
    totalPings = db.IntegerProperty()
    date = db.DateProperty(auto_now_add=True)

class DownTime(db.Model):
    date = db.DateTimeProperty(auto_now_add=True)

You can see from the code above that we are defining three different data model classes - Ping, Day and DownTime.

• Ping - objects from this type are created and stored to the datastore every five minutes. They hold the response time that was recorded when accessing Tutorialzine's homepage. Another property, that is shared by all three data classes, is the date property, which is automatically set to the current time;
• Day - objects from this class are created once per day, at midnight. They contain the total number and average response times of all the pings created in the previous 24 hours;
• DownTime - these objects are only created when downtime is recorded (when Tutorialzine's homepage is inaccessible).

db.Model defines a number of useful methods that become part of these classes. For example saving an object to the datastore involves triggering the put() method:

# This will create a new Ping object and save it to the datastore:
Ping(responseTime=1234).put()

Connecting, validating, sanitizing and storing the data is seamlessly handled by App Engine, which lifts the burden from developers and lets them concentrate on building useful functionality.

The Views

Our views are basically HTML files that are preprocessed by a templating engine. This engine scans the file and inserts dynamic content according to a special syntax, which you can see below. The Webapp framework includes Django's templating engine, which we will be targeting our views for.

As our application is AJAX powered, we need only a single view. Here is what it looks like:

views/index.html

<!DOCTYPE HTML>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>{{title}}</title>

<link rel="stylesheet" type="text/css" href="assets/css/styles.css" />
<meta name="description" content="Uptime and performance statistics for {{domain}}.">

<!--[if IE]><script src="assets/js/excanvas.min.js"></script><![endif]-->

</head>

<body>

<div id="page">
    <div id="header">
        <h1>{{title}}</h1>
        <h2>Uptime Statistics</h2>

        <div id="periodDropDown">

            <span class="left"></span>
            <span class="currentPeriod">Last 24 hours</span>
            <span class="arrow"></span>
            <span class="right"></span>

            <ul>
                <li data-action="24hours">Last 24 hours</li>
                <li data-action="7days">Last 7 Days</li>
                <li data-action="30days">Last 30 Days</li>
            </ul>
        </div>
    </div>

    <div class="performance section">
        <h3>Performance</h3>
        <div id="plot">
            <span class="preloader"></span>
        </div>
    </div>

    <div class="downtime section">
        <h3>Downtime</h3>
        <div class="content">
            <div id="downtimeData"></div>
            <div id="noDowntime">Yey! There was no downtime in the selected period!</div>
        </div>
    </div>

</div>

<p id="footer">
    {{year}} &copy; {{title}}.
</p>

<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.4.4/jquery.min.js"></script>
<script src="assets/js/script.js"></script>
<script src="assets/js/jquery.flot.min.js"></script>

</body>
</html>

This is just a regular HTML file that contains a number of templating tags. These are variable names enclosed in curly braces ( {{variable}} ). They will be replaced with the actual contents of the variables when the template is generated, as you will see in the next part of the series.

The Controllers

There are four main controller files used by the application, all of which reside in the controllers folder:

• ajax.py - this file handles the AJAX requests for showing statistics for the performance of the website. It handles requests for showing data from the last 24 hours, 7 and 30 days. It doesn't output HTML, but JSON, which is used by jQuery;
• crons.py - this is the controller for the cron jobs. It handles two different tasks. Every five minutes it pings Tutorialzine's homepage and saves a Ping object to the datastore, and once per day at midnight, it aggregates the last 24 hours worth of pings into a Day object. You might want to see cron.yaml which we discussed in the previous part;
• generate.py - this file generates sample data in the datastore. It is useful only for testing purposes;
• mainh.py - the main handler, which displays the homepage. It also renders the index.html template using the template engine.

Continue to part four.