Revising the Hacking Den web site.

So a lot has happened in the past three years.

I found myself with a consulting gig that lasted more than a few months. And of course the pandemic turned the world upside down.

My theory with this site, however, was that the things I build–especially transistor-based digital circuits–were essentially timeless, so it didn’t matter how long it took me to build something.

But over the time the project just fell on the back burner.

My interest has never really changed, however. And recently I decided to pick up the mantle, so to speak, open up my copy of Eagle, run some simulations in iCircuit, and start over from the very beginning.

Thus, the revised version of this web site.

Eventually my goal is to start building a table of circuit references to the left, starting from transistors and moving up to a full system.

I also realized video isn’t really my strong point, so I intend to revamp all of the information as a reference web site instead.

We’ll see how this all works out.

Showing the weather.

So here’s a quick little Arduino project which combines an Ethernet shield, a display shield and an Arduino-compatible Adafruit Metro to display the current weather and forecast.

The goal of this project was to pull weather data from a remote server (in this case, the DarkSky API) to display the current temperature, weather and a 7 day forecast.


Now of course you can’t simply just plug in the Ethernet shield and display shield to have them both work together. While both uses the SPI system to communicate, the Ethernet shield uses pins 10 and 4 on the Adafruit Metro for the select line, while the Display shield uses pins 10, 9 and 4.

This means the first thing we must do is rearrange the pins so we don’t have a conflict. The easiest way to do this is to use a prototype shield.

IMG 5388
Since the top sheild will be the display, we can use our prototype shield to shift pin 8 to pin 10 and pin 5 to pin 4. We cut the connector pins and use jumper wires to make it so that pin 8 on the Arduino connects to pin 10 on the shield, and pin 5 to pin 4. A quick modification of the Arduino GFX library will allow us to run both the Ethernet and display shields at the same time.

This allows us to assemble our stack.

Combined


The second problem we have with this setup is that the Ethernet shield does not connect to HTTPS connections as it doesn’t support SSL/TLS. This can be solved by using a WiFi shield which has SSL/TLS built-in.

I solved it for my setup by using a short Java program uploaded to a remove server which performs the HTTPS request and proxies it through to an HTTP request.

In Java, the servlet code simply takes a request, forwards the request to the DarkSky API server using SSL, and returns the results:

/*  WeatherForwardServlet.java
 * 
 *      WForward Copyright 2018 William Edward Woody, all rights reserved.
 */

package com.chaosinmotion.wforward.server;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.URL;
import java.net.URLConnection;
import javax.servlet.ServletException;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

/**
 * @author woody
 *
 */
public class WeatherForwardServlet extends HttpServlet
{
    static long throttle = 10000;       /* 10 seconds */
    private static final long serialVersionUID = 1L;

    static Object syncObject = new Object();
    static long lastCachedTime;

    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp)
            throws ServletException, IOException
    {
        ServletOutputStream out = resp.getOutputStream();

        /*
         *  Determine if we're getting requests too fast.
         */

        resp.setContentType("application/json");

        synchronized(syncObject) {
            long time = System.currentTimeMillis();
            if (time - throttle < lastCachedTime) {
                out.println("{ \"error\": \"throttle\", \"errorid\": -1 }");
                return;
            }
            lastCachedTime = time;
        }

        /*
         *  Pull the request and forward
         */

        String uri = req.getRequestURI();
        int index = uri.indexOf("api/");
        String trim = uri.substring(index+4);

        /*
         *  Make request
         */

        String request = "https://api.darksky.net/forecast/" + trim;
        URL url = new URL(request);
        URLConnection conn = url.openConnection();
        conn.setDefaultUseCaches(false);
        conn.setUseCaches(false);
        conn.setAllowUserInteraction(false);

        InputStream is = conn.getInputStream();
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        byte[] buffer = new byte[1024];
        int len;
        while (-1 != (len = is.read(buffer))) {
            baos.write(buffer, 0, len);
        }
        is.close();
        
        out.write(baos.toByteArray());
    }
}

Compile into a Java servlet and upload to your favorite server and now requests are being forwarded and the results directly returned without modification.


The third interesting problem is this: when you make a request to the DarkSky API, you get back a result that is about 33K in size.

An Arduino Metro only has around 2K of RAM.

So how do we solve this problem?

We take a clue from the SAX parser for XML. The way the SAX parser works is that it scans the input stream, and makes appropriate subroutine calls based on the symbols currently parsed. The beauty of this system is that we can easily discard information we don’t care about, keeping only the information we’re interested in.

In this case, we create a new parser engine to parse our JSON response. the JsonParser class, which performs method calls if we see the start of an object, the end of one, the key/value pair, the start and end of an array, and values in the JSON stream.

This allows us to build a very simple parser which scans only for the information we care about: today’s temperature and wind directions, the pressure, the appropriate icon to display, and the first seven days of the long-term forecast. All of this is done in the DarkSkyParser class.

With that, it’s a simple matter of then getting the weather information from the DarkSky API, and displaying the results. All of this is handled in the main Weather entry point.


The complete sources for the Arduino project is on GitHub.