Hi, everyone. It’s been awhile since I’ve posted and quite a bit has happened, so let’s get started.
When I last posted, the codebase… worked, but it was a mess; everything was crammed into
loop(), making changing one sensor’s functionality a bit of a mess. Sure, architecture can slow your code down, but just a little organization can’t hurt, right?
I figured that I might as well leverage C++ and make each sensor a class implementing an abstract base class
Sensor. I didn’t do much more than that; even the ABC is just a way of enforcing consistent method names. I don’t know a lot of C++, but I don’t really need vtables, range for-loops, and the like for this program anyway.
Doing this made things a bit more manageable; I still want to break up the
update() loop so that I can easily change when or if data is written. This is because, aside from raw data reads and serialization, my main source of optimization after this will be to skip some SD writes.
Skipping SD writes is pretty simple; I keep a running tab of the variance and mean. If, accounting for gravity, it doesn’t seem to have been moving at any point in the data we just collected, we throw out the data and keep track of how many times we did that. When the shark does start moving, we just write “skipped X writes” to the SD card and continue as usual.
Raw data reads and serialization looks a bit more painful. There’s plenty of overhead and cruft in the libraries that we’re using – for example, reading from the accelerometers reads from the registers, packs them into a short int, stores them, and then converts the ints into floats based on the scale. In reality, we don’t need anything but those 6 bytes of raw accelerometer data, and while that example is relatively minor, the benefits stack over time.
Still, doing that requires a lot of investment; low-level code to read the sensors, a new file format to store the raw data, a Python script to convert the data back into a CSV. All that is for benefits that, while significant, are pretty late-game in the grand scheme of things. We’ve got other things to worry about… namely, storing and serving said data for analysis.
Due to the amount of data we’re collecting and the rate at which we’re collecting it, I expect us to be collecting (at the most) about 3 gigabytes a month. That’s not huge at first, but we want to re-use these tags and host them on a server so that no one has to constantly keep a local copy on their hard drives.
One solution we have is to adapt tagbase to our needs, replacing the Microsoft access interface with SQL; we actually know the people who put it together and can get some help from them. There are people who have worked with servers before; I haven’t, and I don’t think anyone has really used SQL yet. This will be a fun experience, indeed…
I’ve got plenty of stuff to do here, both now and over the summer. I got approved for a grant, so I’ve got summer housing both at my college and the Chesapeake Bay… we’ll see how this goes. See you guys in a month or so.