Using an Arduino to verify your web site is running.

We’ve been having problems with our Internet connection, and I also maintain an e-commerce web site which I’d like to make sure is working correctly.

This quick little hack uses the following parts to periodically check that your web site is running:

Now if you look through the specifications for the Ethernet Shield you see it uses data pins 4, 10, 11, 12, and 13. Now data pins 0 and 1 are used during programming of the Arduino–so I went ahead and wired up my LEDs to pins 2, 3 and 5 on my prototype shield:

Schematic

With this built on the prototype board, it’s easy to create a test sketch to verify the LEDs works correctly:

void setup() 
{
    pinMode(2, OUTPUT);
    pinMode(3, OUTPUT);
    pinMode(5, OUTPUT);
    digitalWrite(2, HIGH);
    digitalWrite(3, HIGH);
    digitalWrite(5, HIGH);
}

void loop() 
{
}

I sandwiched the Ethernet 2 shield between the Arduino and the custom LED shield before running the test sketch, just to make sure there was no interference between the Ethernet shield and our custom LEDs.

I used three colors: green means everything is going fine. Yellow means the web site could be reached, but the results returned from the web site were not what we expected. And red means the web site could not be reached.

We can do this by using the Ethernet client and by loading one of the pages off the web site.

Our final sketch works as follows:

/*  NetworkUp
 *
 *      This sketch uses the Ethernet Shield 2 library and some custom hardware
 *  to ping a web site to determine if the web site is reachable and is up. This
 *  repeats for some user-defined period (in practice, about every 15 minutes).
 */

#include <Ethernet2.h>

We need to set up some constants.

The constant INTERVAL is how often our sketch checks the server. The constant below is set to 15 minutes in milliseconds.

#define INTERVAL        900000  /* Every 15 minutes */

The Mac address of the board is located on a sticker on the back of the Ethernet board for newer boards; you need to replace the Mac address constant below with the one on the sticker.

byte mac[] = { 0x2C, 0xF7, 0xF1, 0x08, 0x1B, 0x28 };

And this is the server you’re monitoring. In my case, it’s a web site which sells collectable stock certificates:

char server[] = "hollinscertificates.com";

Next we need a secret string we are checking to see if it is on the web site. The idea here is that since you control the web contents, you can insert a secret string that the client scans for to make sure is present in the return response. This can be a string in the HTTP response header, or a comment in the HTML; all that is important is that if the string is not present, we turn on the yellow light.

And that allows us to investigate in case the web site page is gone (and we got a 404 error), or if our web site was hacked.

char validate[] = "<!-- A Secret Comment -->";

Next we define a number of globals:

EthernetClient client;     // The Ethernet Client
unsigned long nextTime;    // Time (in ms) to check the web
bool connectFlag;          // True if we've connected
bool successFlag;          // True if we found our validate string
byte valPos;               // Position as we scan our string.

Our setup entry point sets up the Ethernet shield. We use certain LED animations in order to help us debug where we are in the startup sequence. All three LEDs are on when the board first starts up and the Ethernet shield is getting its IP address from a DHCP server. Then the lights flash from green to red for a second (in order to provide us the required 1 second delay as the Ethernet board initializes).

We then set a few of our globals.

void setup() 
{    
    /*
     *  Set red, green LEDs on.
     */

    pinMode(2, OUTPUT);
    pinMode(3, OUTPUT);
    pinMode(5, OUTPUT);
    digitalWrite(2, HIGH);
    digitalWrite(3, HIGH);
    digitalWrite(5, HIGH);

    /*
     *  Try to initialize our Ethernet library
     */

    if (Ethernet.begin(mac) == 0) {
        /*
         *  Something went wrong. Fast flash red LED.
         */

        digitalWrite(2,LOW);
        digitalWrite(3,LOW);
        for (;;) {
            digitalWrite(5,HIGH);
            delay(200);
            digitalWrite(5,LOW);
            delay(200);
        }
    }

    /*
     *  Quick animation. This also delays us 1 second to allow the Ethernet
     *  chip to come up.
     */

    digitalWrite(2, LOW);
    digitalWrite(3, LOW);
    digitalWrite(5, LOW);
    delay(200);

    digitalWrite(2,HIGH);
    delay(200);
    digitalWrite(3,HIGH);
    delay(200);
    digitalWrite(5,HIGH);
    delay(200);
    digitalWrite(2,LOW);
    delay(200);
    digitalWrite(3,LOW);
    delay(200);
    digitalWrite(5,LOW);

    nextTime = 0;
    connectFlag = false;
}

Our loop checks to see if it is time to connect to our server, and if so, opens a connection to port 80 and makes a request for our web page.

Client GET requests work by opening a TCP/IP connection and sending a block of text ending in a blank newline. Wikipedia has a very simple example of this. We send two additional parameters besides the “Host” parameter: we ask the connection to be closed when we’re done. And we let the back-end know our user agent name is “arduino-ethernet”; we can use this to filter out our requests from any back-end statistics.

Our connection is preceded by turning off all the LEDs; this gives a visual indication that we are talking to the back end:

void loop() 
{
    /*
     *  At the top of this loop try to get the home page for the
     *  specified web site.
     */

    unsigned long t = millis();
    if (nextTime <= t) {
        client.stop();
        
        digitalWrite(2, LOW);
        digitalWrite(3, LOW);
        digitalWrite(5, LOW);
        if (!client.connect(server,80)) {
            /*
             *  We have an error. Immediately light up red.
             */
            digitalWrite(5, HIGH);      /* Error */
        } else {
            client.println("GET / HTTP/1.1");
            client.println("Host: hollinscertificates.com");
            client.println("User-Agent: arduino-ethernet");
            client.println("Connection: close");
            client.println();

            connectFlag = true;
            successFlag = false;
            valPos = 0;
        }
        nextTime = millis() + INTERVAL;
    }

Note two other things here. First, if we are unable to make the connection, we turn on the red LED. The assumption here is that for some reason we cannot reach the server–either the Internet is down, DNS is not working, or the back-end server is otherwise unreachable.

Second, if we do make the connection, we send our full request (including the blank line); the web server on the other hand listens to the text as it is sent, and a blank line triggers the response. (Think of a TCP/IP connection as a two-way simultaneous conversation; the sender and receiver can send data at the same time. So we need a convention to say “hey, I’m done talking”–and in the case of HTTP, for GET that’s the blank line.)

We then note that we made a successful connection, we haven’t found our match string yet, and we reset the match position to zero. We also note the next time we want to make a request.

Once we’ve made our connection, the field client.available() will be true if a character is waiting to be read. We scan the characters, matching against our magic string–and if we’ve read the entire string, we turn on the green LED and we set our success flag to true:

    if (client.available()) {
        char c = client.read();
        
        if (validate[valPos] == c) {
            ++valPos;
            if (validate[valPos] == 0) {
                successFlag = true;
                digitalWrite(2,HIGH);       /* Success */
            }
        } else {
            valPos = 0;
            /* Catch special case where the mismatch is actually part of the substring */
            /* For example, if we want ABC, and we see ABABC, on the second A, we want */
            /* to actually advance by one. */
            if (validate[valPos] == c) ++valPos;
        }

If we for some reason only get a partial match, we reset our scanner to the start.

We also check to see if the connection is closed. This happens when the web page has been completely served. (Recall we sent “Connection: close” above.)

If we don’t see the successFlag value set, we turn on the yellow LED. This indicates either there is a problem with the web page or the web page has been hacked.

    } else if (!client.connected() && connectFlag) {
        client.stop();
        connectFlag = false;
        if (successFlag == false) {
            digitalWrite(3,HIGH);           /* Did not see our validation string */
        }
    }
}

There you go. A dedicated little circuit that can sit on your shelf, alerting you if there is a problem with your web page.


The complete source kit for the sketch can be found GitHub.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at WordPress.com.

Up ↑

%d bloggers like this: