As Doug warned yesterday, it turns out there is a possibility of the board “chopping” the PIC’s data — timing out mid-read and not getting the whole string. Thus far, I have observed this only with a timeout of 0 seconds, but logic dictates that it is at least possible, though very unlikely, with nonzero timeouts. The greater the timeout or shorter the string, the lesser chance this has of happening, but unless the interval of writing and reading are exactly the same, by Murphy’s Law Freya will eventually chop the data and bork things up.
So, two options are to use a None timeout or read one character at a time. However, Tom is right now figuring out how the PIC will make data available, and this point is likely to become moot, so we’ll put it on hold. What Tom was talking about was (since the PIC will likely have enough memory not to need to break data into many groups) sending two strings as queried for by the board, 3 times per minute. Two strings is a little messier than 1, but certainly workable.
This sort of structure will require more careful timing, so I’ve been looking at python’s sched library. I think it should be possible to run the whole main loop with only one scheduler object per go. The scheduler would be loaded along the lines of:
send Q
in 10s, send B
in 11s, read and process B data, and send T
in 13s, read and process T data
in 15s, save data
in 16s, send all control signals accumulated over cycle so far
in 17s, send all pending emails
in 20s, break loop and reload scheduler
where B and T are for bottom and top PIC groups. Details will be fleshed out when the PIC’s needs are clearer.
I’m thinking a nice clean formatting trick to format converted data on the fly will be to use .format() on a string that specifies the desired format to send to the database. Each data-converting function will process raw data, execute response/error logic, and then format its contribution into the big string. The string can be saved to oneline when done, and this doesn’t require order received to be the same as order outputted for maximum formatting ease. string.Template and substitute() seem to be the easiest implementation…
Another thing that’s happening is that Tom is beginning to draw up a spreadsheet with all data fields, their bounds, their conversion formulas, information, etc. From this, I can continue putting together ConvertData.py. And once I know the PIC’s timing schedule, I can write Core.py and BlockFormat.py. What I can work on now is how I’ll write error-handling and logging, which will probably be the most complex part of this whole operation. Necessary features will include:
- info/error/warning logging into a file on the board
- debug logging that can be turned on or off
- automatic responses to certain engineering data
- emails sent every so often containing alerts
Logging can be handled using the python logging library. Auto-response logic should be case by case, but easy. Control signals generated can be held in a file until time to send them over, and same for emails. What’s not clear is how I can cleanly avoid sending duplicate emails every 15 seconds. Maybe the planned GUI-type control and engineering application will grab those signals after the first email? Maybe I should make the data-fields objects so they know their notification history. In any case, I’ll be back next Monday and know more then.
–Soren