Carleton Weather

A week’s worth of updates for whosoever is interested:

A whole new tower is going up in the Carleton Arboretum, complete with solar panels, ultrasonic snow depth sensor, and more anemometers than you can shake two sticks at. Naturally, I’ve spent the week grappling with such pressing problems as: can we get our TS-4200 to talk like HAL-9000? and should we put a satellite into orbit to transmit the data we gather?

Work on satellites and text-to-speech aside, here’s what’s been going on. The new tower will require a new macrocontroller (Freya, whose cousin Vor I’ve been working on in anticipation of Freya). First project was ensuring compatibility of the various libraries we’ll need. Vor is a TS-4200 mounted on a TS-8200 baseboard running stripped down Debian “squeeze” with Python 2.6.6 (and staying that way) and pySerial. Unfortunately, the RTC does not appear to be functional, which is a bummer, as it means we’ll have to rely exclusively on ntpd to keep system time correct in the case of power loss, and my tests suggest that ntpd can take up to 7 min on startup to sync the system time. I’m thinking right now that the startup script that runs our data collection program should include a call to ntpd, to make sure that happens first. Of course, as Doug has pointed out, if the internet’s down when power is restored ntpd will not fix the time and we’ll have mis-timestamped data. I think the way to fix this is to backfill any data gathered without ntp server access, as per Bruce Duffy’s suggestion. If I do get the RTC working, it’ll be trivial to sync it with ntp-checked systime so that when power returns systime can be set to what the RTC (which is battery-backed) thinks. This would be the ideal option.

I have also been looking into the “end-of-unix-time” bug (short version: 32-bit signed integer holding time_t datatype will roll over in 2038, pushing date back to 1901). Conclusion: the bug is so deep in the debian operating system and even the board’s embedded code that nothing short of replacing the board with a 64-bit time compatible one by 2038 will save us.

Last vor-related thing: hackers are thwarted. I tweaked the host config files to stem the deluge of failed password attempts that were filling our auth.log files (over 10% of available space in a few weeks!) and did a whole lot of other fiddly bits to try and optimize the board for our purposes — saving space and whatnot.

Have been thinking some about the structure of the data collection program, which we’ll start writing next week. We’ll soon also have a PIC to practice talking to, so I’ve written some testing code for that and learned up on serial communication. That’s all!

–Soren (new student worker 1/2)

The struggle is over! It looks like we had the right idea with our code as previously written, but the problem rested with the actual port we were using. It looks like the serial pins we were using were the pins for debugging the code, which is why the data came out so inconsistently when we tested it. So instead of using pins 2 and 3, we had to use pins 7 and 8. It is pretty poorly documented on the TS site, but here are two URLs with diagrams:
http://wiki.embeddedarm.com/wiki/TS-8200-4200#DB9

http://www.embeddedarm.com/about/res…501#section5.1

In short, we needed to quit using the debug port on the the DB9 connector, and use ttyS2 instead.  The way to change ports on the DB9 connector is to use different pins on the DB9 connector.  The resultant code is attached in a Word file because this blog seems to be a little finicky about indentation!

Also, we had been testing our programs from the root user previously, and have now added permission to our wx14 user to execute serial programs. This process is not intuitive, and required some help, which we got here:

The last couple of weeks have been drama packed, kids. Freya is stretching her fledgling TS-4200 wings trying to connect to a PIC chip, but this seems to be too much for her. Or rather her flailing programmer. Thus far, the Python code looks like this:

import serial

import time

ser = serial.Serial(port=0, baudrate=9600, timeout=30)

#ser.close()

ser.open()

#ser = serial.Serial(port=0, baudrate=9600, timeout=4)

print ser.portstr

#c = ”

#line = ”

for i in range(10):

#ser.open()

c = ”

line = ”

#ser.open()

ser.write(“Q”)

print ‘sent q’

while c != ‘\n’:

c = ser.read()

if c == ”:

c = ser.read()

line += c

#ser.close()

print line

time.sleep(2)

ser.close()

##print line

#ser.close()

(with better indentation and coloring.) The results are mysteriously varied; the goal is to get 10 strings with differing values, and the result is not that. Somethines we get 5 or 6 such strings before it times out, most times we get no strings at all with a what seems to be a serial port confusion error (like it was previously opened, was never opened, etc), and otherwise we get some middling combination. The struggle shall be continued later on!

This weather system is a never ending onion, so ridiculously layered! Everyday I set an objective for learning about some aspect of the system, which I am still embarrassed to admit that I was completely unaware of until too recently. I’m making up for this by excitedly telling fellow Carls to use it, however. After getting the TS4200 on the network, I’ve mostly been shipping software over to it and figuring out how to play with a computer which has no friendly GUI. Luckily, the documentation left for me by my blogging predecessors is hugely helpful. So much so that is it even easy to figure out what has been missing as we must fill in with what I’ve learned a bout the TS4200 so far.

Maybe the title was a tip off to the change in management of this blog, or maybe it was the elapsed two years, but as Chance the Rapper proclaimed and I lifted for this title, WE BACK. By which I mean the Carleton Weather Station once again necessitates an assistant, which is now me, Anne! See profile for more details.

This is my second week on the job, and thus far a majority of my work has involved familiarizing myself with the massive arsenal of equipment and software involved in this project. To be honest, I had no idea that Carleton had anything to do with weather before applying for this job, and indeed it does! And does it ever. The word is that Carleton has the oldest weather records in the state of Minnesota, dating back to 1881. Additionally, I just learned today that Carleton used the instruments housed at the Goodsell Observatory to set an accurate time for the region, to which the Western US subsequently synced their clocks. Which sounds to me like quite the big deal.

The methods then used for doing so included sextant use, a skill which I possess and never once thought I would have to call upon again. The sextant is a beast of an instrument, measuring the angle from the sun to the horizon. With the use of a couple of different almanacs and the knowledge of your degree of latitude, the exact time can be calculated! But let me tell you from experience, it is quite the calculation involved. I recently completed a SEA Semester, where we put this skill to use on the high seas, our efforts justified by the phrase, “We’re all one fuse away from Columbus.”

Currently, Carleton has several weather tracking instruments in the field that measure all of the neat things reported at weather.carleton.edu. These devices report to a Unix-based machine which records the data, and ships it off to a connected Mac which updates the aforementioned weather page. Updates are always welcome and necessary, so that’s what I’ll be working on for my summer here! I’ll keep all you readers in the loop in the meantime, thanks!

Anne

Wunderground protocol for uploading data to a personal weatherstation

http://wiki.wunderground.com/index.php/PWS_-_Upload_Protocol

—-

Two radar animations (perhaps for our site), courtesy of WeatherUnderground:

http://icons.wunderground.com/data/640×480/mw_ir_anim.gif

http://icons.wunderground.com/data/640×480/mw_rd_anim.gif

—-

From Tom:

I found these bits of information on the weather exchange file formatting:
http://ambientweather.wikispaces.com/Weather+Exchange+csv+File+Format

http://www.thunderheadtech.com/WxSolution/   (scroll to the bottom)

This tool, for a Mac, looks like it does what we want for us:
http://www.weathersnoop.com/features/manage.php

This page says you can view your weather from your web browser – I wonder if that means anyone could if it is set up that way:
http://www.weathersnoop.com/features/share.php

This page shows the weather App:
http://www.weathersnoop.com/features/observe.php

Well, looks like this is it. I uploaded all my code to COLLAB in the folder weatherFinal along with my documentation file. It’s been lots of fun, feel free to give me a holler if anything breaks or something is really poorly worded.

-Will

I got the changes we talked about yesterday implemented. Some of them work, some of them don’t, and a few are questionable. Here is a summary:

• Added dawn/dusk and the cannon river link to the homepage
• Added a bar to the section titles on the homepage (not sure I like this)
• Tweaked the layout of the plot form so that it is not quite as long
• Added a note about incomplete data to the Almanac (this needs to be re-styled though)
• Fixed the weird year issue with the plot page

I think that is it, my list now resembles a bunch of scribbles with Xs through them.

Well, I got the time thing to work, i’m not really sure why it works but it does. I made a backup in case we want to revert. Also got the nav bar finalized and made some headway on my documentation for the tabs. I’m heading up to the cities this weekend to coach, so if you need anything shoot me an email or call me.

-Will

The whole unixtime problem from the first page gave me such a headache by lunch that I put it away for the day. I think that there is some small detail in the PHP functions that I am missing. This afternoon, I started working on the date adjustment bars for the almanac page. I have them in, but they are not properly styled and there is a small bug that allows the user to request pages in the future (which just show up blank, so it is not a huge deal).

-Will