Saturday, August 2, 2014

DNT900 / RPI900 / Remote Weather Station lessons learned

Debated breaking this up into a bunch of little posts, but I tend to slack on this blog, so here 'goes a smattering of lessons learned while working on the Remote Weather Station Project.

The weather station has been in the field for several months, and I've learned a few good lessons to share. I could imagine these issues/concepts could be applicable to any remotely controlled and/or solar projects. Some solutions to remote projects are provided specifically for the rpi900/dnt00/RPi, but these items issues should be considered if you are looking into other communication or power management devices.

 http://rpi900.com/ remains the best source for working with dnt900 in conjunction with the  Raspberry Pi (and perhaps 2nd to the dnt900/dnt2400 manuals themselves).  Also it is very exciting that Matt has made the various rpi900 software components into Arch Linux packages. That said, here are some lessons learned on my specific project:

Link Stability of PPP over DNT900

My remote Raspberry Pi can be SSH'd into, and communicate with Xively and to push its measurements. My RF connection between my dnt900 radio's supporting this (PPP network) is very much on the edge of its capabilities, the RSSI is frequently near -100 dBm. Shortly after getting the weather station in the field I noticed many of my Xively posts were getting dropped (as the dnt900 base/remote PPP network was unstable.) Ping showed extreme delays, and the packet loss was high. Poking around in the journal, I found a way another way to check for radio connection stability.

[jay@basepi ~]# journalctl |grep minute
basepi pppd[332]: Connect time 1.2 minutes.
basepi pppd[332]: Connect time 0.4 minutes.
basepi pppd[332]: Connect time 0.1 minutes.
basepi pppd[332]: Connect time 3.9 minutes.

So now that I was keenly aware of my problem, I was pleased to find out the dnt900 radio's had some handy features to help me solve my problem.

The radio default for each packet is to only try 8 times. Bumped up to max retries (63 I believe). Radio's now remain connected for days/weeks at a time.

echo "0x00" > /sys/class/dnt900/ttyAMA0/0x000506/ARQ_Mode
echo "0x3F" > /sys/class/dnt900/ttyAMA0/0x000506/ARQ_AttemptLimit

On a related subject, I found a ligherweight SSH setting was more tolerant of the poor connection. I eventually didn't feel the need to use, but was impressed that it was helpful when the network was spotty.

ssh -v -C -c blowfish <hostname> 
-v optional

Power Management

When I initially deployed the system, my power usage was very close to my consumption, so my battery was more apt to drain if my use of the radio link was excessive. Given the fact I extended out the Max retries for each packet I also suspected the transmission length of the radio was challenging to characterize and control.  I initially spent a fair amount of time deciding on the intervals of the data posting (trying to find the minimum I could live with.)

At some point I stumbled across a post on power conservation on Raspberry Pi. (I unfortunately cannot find my source at the time.) I learned (albeit late) Model A, vs. B, would have made more sense in my configuration. Some options existed to make the power usage on a B similar to A, but they do have significant implications.

The HDMI output can be disabled on the Raspberry Pi (minimal power savings); I don't use the HDMI on my battery powered Pi,so this was a no-brainer:  Here is the command I use to remotely disable, same command could be run locally on the remote Pi:

ssh jay@remotepi '/opt/vc/bin/tvservice -off'

The other option is to disable the usb/buspower. This would make USB ports and the Ethernet ports unusable. I'm not using USB, so I gave it a try. I was pleasantly surprised to realize I didn't need this powered since I'm using PPP (over serial/ttl/dnt900). Will not be applicable to most applications, but worked in my setup. Again, a command to remotely disable:

ssh jay@remotepi 'echo 0x0 > /sys/devices/platform/bcm2708_usb/buspower'

With bcm2708_usb buspower removed (and perhaps hdmi), my power usage profile changed dramatically. My solar setup creates and stores much more power then I need. I'm feeling confident it may be able to run without (any) sun for more than a week. (My panel is already positioned for winter sun.)

In case you missed it, the RPI900 has an excellent ability to Remotely Monitoring battery levels. Has been critical to my project. Was very pleased that was thought about ahead of time in it's design. I post my battery level to Xively, its awesome to see the history, and use that data to see effectiveness of  adjustments to the system.


Soil Moisture Sensor 

The moisture sensor chosen for this project was regrettable (SHT10/SHT11). Long story short, it is expensive and not really helpful for measuring moisture in a way a gardener might find helpful. ( Details here, perhaps only relevant if your shopping around for Soil Moisture Sensor Solutions.)


Heat Management 

In prior posts I walked through the rather extensive steps take for  heat management  of the remote garden station. That has been working out great. The station (in a small, weather proof case) has been outside with temps above 105 °F (40.5 °C), and Raspberry Pi CPU temps maintained below 45.0 °C. (As a point of comparison, my base station which exists in my attic on similar days gets north of 50.0 °C.... and it actually has a fan blowing directly on it.) I see a fairly wide temperature fluctuations (+/- 25 °C in  day) on the fan-cooled Raspi base station vs. the  heat sink/pipe cooled remote station  (+/- 10 °C in  day).

The heat sink/tube setup was a project unto itself, so a primary drawback of it is that if I wanted to upgrade/replace the station Raspi (to say a B+, or change to a model A) it has large implications.


Misc Items

Noob: If built another station (or perhaps any Raspberry Pi project) I would likely skip the NooB option and directly load the OS of interest.  I started with a 4GB card, and lost a bunch of space by 'trying' other OS. I now have to keep a closer eye on my disk space then would be desirable.

Kernal update: On  a couple occasions I performed a full update on the Raspberry Pi. Each time the dnt900 line discipline required rebuild. The first time I encountered this I lost some time trying to figure out why the radio's stopped working.  Luckily Matt  (behind RPI900) was nice enough to steer me in the right direction.

Testing configurations:  Before I had the units in the field I needed to be able to directly log into each Raspberry Pi (and not over the RF PPP link). Get the sequence of event right to allow this ultimately boiled down to disabling ppp if the dnt900 line discipline was required on the remote.

Updating remote radio configurations: Obvious in retrospect (but I did cause myself some heartache with this a couple times) Long story short, if updating remote radio settings (such as Hop Rate/Data Rate, ect. ect.) be sure to update the remote radio first (expect the link to drop), then update the base station. When radio's reconnect, the values can be made permanently stored values.

Conclusion

The project has been a fun one, but I'm still debating as winter approaches if I'll leave it in the field.  Perhaps taking a 'run-to-death' approach of managing it, just doing triage and preventative maintenance. Or, I may pull it indoors for upgrades. At the moment leaning towards just preventative maintenance, run-to-death, to focus on other projects (to include the garden itself.)

Final thought on this project is that  all the weather/garden data in the world does not make for a more lush and plentiful garden. I spent a fair bit of time early in the season working out the weather station issues, and the actual garden suffered for it. Hmm,  maybe some water automation could help that.... 

Friday, August 1, 2014

3D Spectrum display in Chrome with rtl-sdr


Saw an interesting post on rtl-sdr.com about a 3D display using rtl-sdr from the browser. The github repo found here, and the original source page is a Japanese one found here.

I was impressed with how straightforward it was to use, and on my Windows 8.1 i5 it ran well. I would say the video doesn't do it justice and my immediate reaction was simply, wow.

 If I was to just going quickly run the rtl-sdr to see what is the RF situation is, this may become my go-to. I look forward to seeing a few more features, and see some of the minor usability and bugs worked out, but it is a great start.


Monday, April 28, 2014

The case build (for the weather station), and installed in the field

Around the time I started to plan for the weather station case I came across a post on reddit that peaked my concern about heat management of the Raspberry Pi. Further reading on HackaDay confirmed it was a factor to consider. My RPi weather station is not stressed from a network or CPU perspective, but the summer heat on the sealed case could be show-stopper. From what I've read, 80ÂșC is the critical temp for the RPi. (Link also includes a shutdown script if critical temp is approached, something I may consider implementing.)

What follows is  overview of my case build dealing with both weatherproofing and heat management for the RPi in my remote garden weather station.

For starters, below is a quick review what is in the weather station (case), or external to it. Pictured below are the core components of the station: a remote and base-station. Both include a Raspberry Pi (RPi) with the RPi900 add on board hosting the DNT900 radio and antenna. (The WiFi pictured not used in the final system, only for testing.) The base-station for the system remains in my home, one of the units (the remote end) goes in the case installed at the garden ~2 miles from my home.


Aside from the RPi, the case also needs to house the electronics that interface with the RPi's GPIO. I discussed all the sensors in a prior post and bulk of the board is show below. What is missing from the image below are the sensors attached to the breadboard.



The RPi and breadboard above share a similar footprint, so I ended up picking the Otterbox 3500 for the final case  (better to Google this), it is not produced any longer but deals can be found on Ebay and even Amazon. The Otterbox by design is naturally waterproof. Although I planned to add multiple holes for sensors and power I felt it would be better to start with a waterproof case, then try to make a traditional case water-resistant.

Below is a rather boring picture, but what can be gleamed from it is (and the picture that follows it) is the relative size of the RPi in the case. You can also see standoffs I placed inside the case. The standoffs hold a copper plate slightly elevated from the back of the case to allow for other mounting items, and cables to be run under the plate.



Below is a RPi  in the case, on the copper plate (actually a bad/broken RPI, one I just use for test fittings), as to not rough up my good hardware).


The mount (to attached the case to a wooden pole) being added below. Back of case seen.


The inner attachments of the mount (The nuts do not protrude higher than the copper standoff plat.) Case is actually up-side down in this pick .


So back to the copper plate (mentioned previously.) At fist mention, that may have raised a few eye-brows. The backing copper plate is actually multi-function. One purpose is simple to ease installation mount everything to a plate, then just insert the plate inside the case. The more significant role is actually part of the somewhat elaborate cooling system. Anyone familiar with RPi has likely seen the heat-sinks (of questionable value) that get taped onto the RPi, often made from aluminum.

With my soon to be sealed/waterproof case, I anticipated that simply sticking on heat-sinks wouldn't allow the heat to escape the case , and a cooling fan would be impractical. (Size and power limitations of my solar installation.) My solution was to somehow get the heat-sinks to facilitate heat exiting the case. In comes the idea for a heat pipe solution used to cool laptop computers. My implementation is not a perfect heatpipe setup, but hopefully enough to get the job done.

I chose to go with a full copper system (for heat-transfer properties, and to simplify the soldering/adhesive needed to hold it all together. Below you can see the start of attaching 1 RPi heatsinks (copper) to copper heatpipe, and to the copper backplane. 2 more were done like this and later bent into place.



Next you can see the RPi mounted on the copper backplane (and the 3 heatpipes/heat-sinks not yet attached.)


Now a few pictures outside the case, this time withe heat-sinks in place (not yet attached) and the soon to be external heatpipe/sink that will help transfer heat outside the case.

The heatpipe/sink that will be external to the case is from a Toshiba laptop, used to cool a Tecra M3 (I believe.) Found with some heavy Ebay searching for something just right the right size and shape.



The following few pictures now show how a Dremel was used to create the access points to the case (and external heatsink).  The Dremel was perfect for the job. It spun fast enough to not only cut the case, but also slightly melt the plastic and make smooth access points.


External heat sink


View prior to attaching to backplane


Test fitting of backplane and external heatsink


Near final fitting, RPI in-place with heatsink.  Since soldering the external heatsink was not possible in the case, Thermal Adhesive (available from Amazon) was used. Thermal Adhesive was so convenient if I did it again, I might have chosen to use that to attach the heatsinks to the tubes connected to the backplane.  


Slightly out of order, but hear is a view of all the cut-outs (all part of case that faces bottom) for sensors and antenna feeds existing the case as well power inputs. (Largest hole is power, followed by air sensor, battery and soil sensors, then the smallest for the bulkhead SMA antenna connection. Flat hole is heatsink discussed above)

Ok, now is the unfortunate time in the post (and project) where I got very excited about how things were coming together and spent very little time taking pictures, and more time assembling the case itself. So here is a near final view (in outside on my deck being tested.)


An now with the case opened up. You can clearly see the the backplane and (white) thermal adhesive that attaches the external heatsink. What is not obvious is I added some additional spacers between the RPi and the RPi900 radio to increase the air-space around the RPi. The heattubs also don't block the camera, ethernet port or SD card incase access to those items are ever needed. The RPi900 can remotely control power to the RPi but I chose to also add a switch inline. The GPIO headers extend through the RPi900 and the GPIO extension cable sneaks under the backplane before connecting to the breadboard. 1 air temp sensor remains in the case, 3 extend out. (1 for actual weather monitoring, 1 for battery case temp, and one for soil temp) PG


Here is the whole system in a deck test. (Solar, battery and weather station) Even had some rain and freezing weather to test it out.


Fast forwarding abit, here it is installed at the garden. You can see I have added a camera to the case, painted the case white (to reflect heat), as well as added another standalone heatsink to the DNT900 radio.


A nice clear shot of the bottom of the case


A view from below


Looking right at the case and into the camera that peeks out


Just a nice shot of the solar panel and case


The whole setup in the field.


The project is nearly complete, and sensor data can be seen online at http://mygardenspy.com/ (Unless it happens to be offline for maintenance). A few posts will follow on the subject to include more lessons learned, problems encountered (and worked through), and hopefully a single post summarizing the project.

Tuesday, April 22, 2014

DECT 6.0 collection with HackRF

After seeing an interest in DECT on reddit  thought it would be easy enough to grab a recording of this and share.

I have a vTech DECT 6.0 cordless phone system.  I may ciricle back and update this post with more details on DECT and the phone itself, but for now wanted to quickly share the results

Some initial observations: 
  • Frequency usage isn't as high as I thought it would be (number of frequencies), I searched around abit and it seemed 80% of the traffic (as least was isolated to two frequencies)
  • The density of transmissions seemed highest at initial call setup, and on hang up. (But this may be due to the display on my phone receiving information from the base station.
  • Both the handset and base station are visible in the file.  I believe the stronger signal is the handset.
Below is a 8MHz collection via hackrf_transfer as follows:

hackrf_transfer -a 0 -r 1924MHz_DECT_hackrf_8MHz_8MS_8bit.iq -f 1924000000 -s 8000000

Collection was focused around this frequency since this seemed to be where my phone was primarily active. 




And DECT waveform zoomed (in time):




Finally, here is the sample  of the DECT 6.0 from my vTech cordless if you want to take a look yourself:




(To view in Baudline, decompression off, Initial byte 2, sample rate 8000000, channel 2 quadrature, 8 bit linear, unsigned)  NOTE: these collects from older HackRF firmware, latest firmware creates signed 

Saturday, April 19, 2014

Baby Monitor and WiFi HackRF Collection

After breaking out my HackRF to check my DNT900 radio I thought it would be interesting to check out a system I wasn't very familiar with.

I own a couple Motorola Baby Monitors. They advertise as using FHSS, but this time in the 2.4 GHz range (vice my DNT900 that utilize the 900 MHz ISM band.)

Looking around the 2.4 - 2.5 GHz freq range with the HackRF using GQRX I could see there was alot of frequency contention (activity), so isolating the baby camera (& monitor handsets) could be a challenge. In a successful attempt to find some radio silence I powered off my cordless phone, WiFi (IEEE 802.11g) router, put all mobile phones in airplane mode.

Attaching an antenna for the wrong frequency band (co-locating the monitors near the HackRF) further eliminated interference from neighbors 2.4 GHz emissions. 

I also spent some time experimenting with the handset and monitors to see the RF activity in the various audio and video modes. 

Some observations: 
  • Sending audio and video (default) keeps these radios very active.
  • They only send data when the monitor is tuned to them. (I believe) which is interesting, since this may be a way to see if anyone else is snooping on your monitor. If you can set up a passive RF monitor like hack-rf or rtl-sdr.
  • I do not know if they can send to more then one monitor at a time.
  • Cameras and monitors are uniquely paired. (But like anything with security, if someone has access, that is trivial to establish.)
  • The radios use the entire range utilized by WiFi, Bluetooth and countless other commercial products available. (So they are a potential source for degrading WiFi, and from their activity levels I'd be shocked if that is not happening.) 
  • The data bursts are short (< 1-4 ms), and more narrowband than WiFi.

Below is a 20MHz collection via hackrf_transfer as follows:

hackrf_transfer -a 0 -r 2417MHz_motorola_hackrf_20MHz_20MS_8bit.iq -f 2417000000 -s 20000000

Since this is only 20MHz of the possible ~60 MHz the radios use not all data from this ~1 sec timeslice is shown.



And Motorola waveform zoomed (in time):





Look close (on the 1st image showing the larger time scale and you may see two, since two baby monitors in use. Thinking the other just announces its presence when not tuned to (for the scan option.)

Since the manual for these devices focuses on what parents need to know, and not those curious about how it works, alittle FCC database research was needed.

Here is the site for that:

The camera and monitor and manual all had FCC Id's listed (as do most legal equipment). For unknown reason, directly plugging that information in did not work.  I ended up just using the Grantee code (first 3 letters), and providing a frequency range.  That narrowed it down to 10 items easily reviewed.  Clicking "Display Grant" gives more summary information to find the device of interest.

Looks like Binatone Electrons actually makes the radio itself (not Motorola), and they are a big supplier to Motorola, as well as a wide range of other products that require some form of radio.

For example, Camera Unit FCC Grant:




It does seem for this equipment the manufacture requested confidentiality, which didn't seem to be honored (best I can tell.) But I'll at least not re-post anything from their reports and let you do your own exploring. Test Report being the most interesting for those wishing to understand the system in more detail.

Or maybe the internal photos to learn about the device without cracking one open

Finally, for comparison purposes. Here is a collection at the same frequency & Bandwidth that just shows alittle bit of WiFi activity, but baby monitor turned off.  (Off-tuned for WiFi, but I think it still is clear they are very different waveforms.)


And WiFi (IEEE 802.11g) waveform zoomed (in time):



Finally, here is the sample  of the Motorola Baby Monitor transmission if you want to take a look yourself:

2417MHz_motorola_hackrf_20MHz_20MS_8bit.iq

(To view in Baudline, decompression off, Initial byte 2, sample rate 20000000, channel 2 quadrature, 8 bit linear, unsigned) NOTE: these collects from older HackRF firmware, latest firmware creates signed 

DNT900 Radio Transmission Collection with HackRF

Since I happen to be a lucky owner of a HackRF I thought it would be interesting to collect the transmission from the dnt900 used in my remote weather station project. Here is a 20MHz collect using hackrf_transfer

I believe this recording captures most of transmission (primarily my base station), at 38.4 kb/s data rate.

For those looking for nitty gritty details the FCC Exhibits lists is somewhat interesting, not much more information than found in the devices users manual though:
https://apps.fcc.gov/oetcf/eas/reports/ViewExhibitReport.cfm?mode=Exhibits&RequestTimeout=500&calledFromFrame=N&application_id=241168&fcc_id=HSW-DNT900

Here is a link to the recording if you want to check it out:
915MHz_2_dnt900_hackrf_20MHz_20MS_8bit.iq


If your using Baudline to view the HackRF collected files (via hackrf_transfer)  here are the settings:

NOTE: these collects from older HackRF firmware, latest firmware creates signed 


Thursday, April 17, 2014

Weather Station data displayed on public website (Custom Xively-rickshaw/plot.y and Google Gauges displays)



Last post I highlighted getting all my weather sensor data available on Xively for my RPi hosting sensors accessible remotely over an RF link.

While I think Xively is great, it is really not designed to be the final stop for sharing or exchanging data. My weather station data is now consolidated at http://mygardenspy.com/ Displaying the data on my own site allows for more customization, and presentation of the displays, and easier access for my fellow gardeners. The page is broken up into three sections right now, current garden conditions (what most care about), hardware status (how I keep an eye on my equipment running), and historical data.

The skeleton of the site (I'm not a web developer) pardon my naivety, is Twitter Bootstrap.  I'm just a general fan of that toolkit, to include the documentation. Their templates seem fairly easy to grab and modify. Also looks decent on desktops and smartphones.

Current Conditions 

I originally had high hopes of uses a wide range of current  Google Charts but had a tough time figuring out how to use the Xively JSON. Seemed like I might be fording new territory, not entirely a productive activity for someone with limited web experience.

Luckily I came across a great blog sdyoung.com that had figured this out for the Google Gauges, so I used this extensively (on both the Current conditions, and Hardware Status.) If you want to check out my derived work, check my github.

The line graph seen below the Gauges is actually a image retrieved from Xively (for simplicity). Discussed in my last post  But also here is the basics of the html again. 


 Graph can be adjusted by following the Xively API.


For the historical data page I wanted something interactive. I came across this post about plotting Xively data with RickShaw. Was exactly what I needed, and well documented at the xively-rickshaw github. I use Xively-Rickshaw for nearly all my historical displays, both line graph and bar charts. Here is what I implemented from extensively referencing (mostly copying) xively-rickshaw example. Although the Xively feed does restrict the number of points that can be pulled, that model fit well into my various historical data view options.

On the RPi side my various scripts to pull the data are started on boot and otherwise controlled via supervisord I found its control and logging, and able to background processes and control sleep withing the scripts, not under the restrictions of cron useful.

Modifying /etc/supervisord.conf Adding something like the following:

directory=/root/base_monitor/
command=/root/base_monitor/post_rssi_xively.py
autostart=true
autorestart=true

Enabling the service (or later restarting/stopping) is as easy as:
systemctl enable supervisor.service | systemctl restart supervisor.service | systemctl stop supervisor.service | systemctl start supervisor.service

I should mention, I am using Arch Linux,  so my supervisor is first called at startup by systemd:
/etc/systemd/system/supervisor.service 
Example:
[Unit]
Description=control progs
After=ppp-remote@.service
After=systemd-modules-load.service
[Service]
ExecStart=/usr/bin/supervisord
Type=forking
[Install]
WantedBy=multi-user.target


Aside from xively-rickshaw I did some experimenting with plot.ly. At quick glance, it might look similar, but plot.ly does offer some interesting features such as zooming  in, or going into plot.ly to fork (copy and analyse). I haven't explored it much, so I haven't figured out if there is an easy way to display smaller periods of data (like I do with Xively.) I use it to display all CPU data from my base station (from inception.)


For my RPi and page I came across a simple method that worked great. It relies on cron at the moment (on my RPi). There are also some examples of more advanced use of plot.ly if you care to look more into this method.

My full base station plot.ly CPU temp feed: https://plot.ly/~geekskunk/5/ 


Well, that covers it. For the all the details of the web page side of things check out https://github.com/geekskunk/remotegardensite/tree/master/public_html






Friday, April 11, 2014

Getting Sensor Data online with Xively


I was able to recently prove the sensors for temp and moisture (using the SHT11 & DS18B20) worked well with my RPi setup. As I mentioned in the original plan, some early research indicated that uploading the recorded data to Xively would prove useful for later public web access, and help me avoid constructing my own database for the data. 

I'd like to make the weather station data publicity available for my fellow gardeners, but do not want any dependency on the sharing of the data on the RPi itself. In my setup using PPP for my remote radio I intend to have the remote put the data to Xively via my base stations physical network connection. So far, in local testing (around the house) it is working well. Below describes how I've got it working. (Referencing the great work of other.)

I'm not going to detail the steps to get an Xively feed setup, and data posted to it. I followed others tutorials online that did a better job than I'd do. The one provided by Xively itself ended up the basis for nearly all my feeds: https://xively.com/dev/tutorials/pi/, and for those looking for the nitty-gritty details, I'll save you tracking down the API:  http://xively.github.io/xively-python/, and the wrapper library itself: https://github.com/xively/xively-python   AdaFruit (as always) has a good follow-along tutorial that could also be referenced: https://learn.adafruit.com/connecting-the-max31855-thermocouple-amplifier-breakout-to-an-electric-imp/configure-the-agent-for-xively

Here are all my scripts for grabbing data, as well as the scripts for posting to Xively: https://github.com/geekskunk/remotegardensite/tree/master/garden_monitor Update your API and channel names accordingly if you borrow.

Pictured below and available here is my Xively Feed: https://xively.com/feeds/1824193320  I find this Xively page helpful for general use, but for a wider audience a more refined site is needed (and in the works.)



Now that I have the data steady flowing to Xively, I'll work next on visualizing the data in a more useful way on my one site. At the moment I'm considering: https://xively.com/dev/tutorials/channel-viz, as well as this https://jasonneylon.wordpress.com/2013/05/26/plotting-xively-timeseries-data-using-rickshaw/ Also for quick view of current values Google Gauges, like from this tutorial: http://www.sdyoung.com/home/home-status/how-i-visualize-the-sensors/jquery-and-json-enabled-google-gauges/

Until my site is up, here are a few quick pointers for pulling the Xively data. They are not interactive, but do provide an alternative the Xively site proper. I was tipped off on this simple approach from this site which extensively uses this method.

Actual html:

<img src=https://api.xively.com/v2/feeds/1824193320/datastreams/air_temp.png?w=800&h=200&c=2188c5&g=true&t=Last%2024Hrs%20Air%20Temperature%20History&b=true&duration=1day&scale=Datastream&timezone=-05&l=Internal%20Air%20Temperature%20%C2%B0F>

Template html:

<img src=https://api.xively.com/v2/feeds/INSERT_YOUR_FEED_ID_HERE/datastreams/CHANNEL_NAME.png?w=800&h=200&c=2188c5&g=true&t=INSERT_TITLE_HERE&b=true&duration=1day&scale=Datastream&timezone=-05&l=INSERT_YOUR_LABLE_HERE>


Example output:

For more API details (which apply to the image generation) check out: https://xively.com/dev/docs/api/data/read/single_datastream/




Wednesday, March 19, 2014

Perils of a Part Time Project (and some solutions)


My sensors in the Remote Garden Project stopped working unexpectedly while I was away at work recently. After a few minutes of poking at my putty terminal I decided to inspect the actual setup. The breadboard and pile of wires was a much different pile of wires than the last time I saw it.  I thought I had it tucked away inside my entertainment center where it wouldn't get disturbed by anyone.....I was wrong. My wife found her way in the cabinet, and some of the wires poured out; with the best-of-intentions she pushed the pile o'wires back in without any discrimination. Needless to say that maneuver does not make for well functioning breadboard setup.

After explaining that the pile of wires was an intentional setup (and I had to re-wire everything)  it became crystal clear I needed to formalize and document my setup abit. I could also understand her point of view, it was a pretty big mess. It also certainly needs some finishing to get into the final weather station.

For reference, here is the what the "before" setup looked like on the breadboard:


After a bit of searching I came across a solution that was just what I needed: http://fritzing.org/ Really sweet software package that I'm just barely taking advantage of. It is nice visual breadboard capability with a ton of parts that can be dropped in an customized. As you breadboard out your design it also makes a schematic and PCB (that you can have them Fab for you.)  I've only used the visual breadboard feature so far, but it will be something I consistently use in the future.

As you can see in my fritz prototype below I was able to work on the layout to get all my sensors in a somewhat logical layout. Unfortunately the exact GPIO expansion/breakout board wasn't mocked up, so I crudely created the connections directly from the pi.



Slightly more order to the chaos, and gave me some more confidence to move to a protoboard.  My GPIO breakout came with a half sized breadboard so I went in search of protoboards that could be swapped in easily.  Adafruit had some great looking ones, but turned out they didn't fit the power rails of my GPIO expansion/breakout at all. (They looked great, but left me searching for a better fit.)

 The near exact match ended up being the SB400 from busboard, ordered from Mouser. With that part in hand my setup is now really starting to come together nicely. No longer a wild pile of wires!


So with a part time project like this it was really helpful to sit down, plan and organize abit. Fritzing was a big help, and even allowed me to revisit and test alternate sensors setups without purchasing anything which is also pretty sweet.

The other helpful tool for keeping on top of this multi-faceted project has been a decent to-do list. Seems simple enough; but I wanted it to be segregated my personal task list. To make a clean distinction I decided to use GitHubs issue tracking tools for this purpose. I've already backed up much of this project to GitHub, so why not the task list as well. With custom labels in help separate software bugs from general task, or work to do on the hardware like the solar setup. 

Tuesday, March 18, 2014

Sensors used in the weather station


Spring weather is fast approaching, so I have started to focus more on the sensors for my remote RPi Weather Station. For simplicity (and initial proof of concept purposes) I decided to utilize just digital sensors (i2c) rather than attempt analog. Analog looks possible, either this method, or perhaps ganging together a Arduino. Perhaps something I'll consider in the future.

My search for i2c sensors was somewhat two-fold: does it look to be a digital sensor (i2c compatible with some history of prior RPi integration), and is there some example code to leverage. Seemingly a low bar, but it did narrow down the options pretty well. I tend to be a fairly loyal customer, so adafruit got the brunt of my business.

The primary temperature setting I ended up getting was the weatherproof  DS18B20. The sensor being waterproof was selling point, a great tutorial, and the fact that each has individual ID, so I can have multiple of this type. (At the moment I am thinking I will use one for air temperature, internal case temp of the weather station and perhaps temperature of the battery case.)

For monitoring the soil I went with a weatherproof variant of the SHT-10 The SHT-1x seem fairly common for weather sensing. Adafruit highlights usage with Arduino, but given my limited options for soil moisture and temperature I went this route. Looks like only 1 of these sensors may be possible in a individual system, but that limitation didn't bother me much for this sensor.  I also found another resource for reading the data from an SHT-1x sensor.

So if you've hung in this far you see I really only have two external sensors in use for this weather station at the moment. Since they are i2c (and I already have a rpi900 board attached to my RPi) I needed to pick up extra long headers. If I had more foresight I could have saved my self some de-soldering/soldering when replacing the headers on the rpi900; so on the off chance your copying this exact project, hopefully you catch this early.

Perhaps, since I was in the midst of ordering diapers for my little-one, I ended up picking up a GPIO breakout/expansion kit for the RPi from Amazon instead of a more conventional electronics source. I grabbed the GeauxRobot GPIO Expansion kit which came with the GPIO breakout board, an extension cable, and a mini breadboard.  (GeauxRobot seems to have an Amazon storefront as well as a conventional site, but I've only ordered via Amazon, personally.) FWIW, in the past I have also ordered a couple different RPi cases, a GeauxRobot and Kootek. GeauxRobot case was a dollar more, but much more solid and better built.

When I had all my parts on hand going from pile-of-parts to air and soil moisture sensors was made easy with help of some of the links I posted above. I was also pleased that they inter-operated with each other, and the rpi900 board.

To get the Raspberry Pi and sensors working together I used this exact setup, the adafruit learning system (team) also included a Python program to read the temperature data. My final slight modifications include making the scripts for each different DS18B20 unique, and only writing the Fahrenheit temp to a file instead of to the terminal.

For the soil and moisture sensor I again borrowed heavily, this time from John Geek way-down in NZ. (Also see his updated link on the subject.) John provided a great overview of the wiring, as well as C code for reading the data. For the wiring I also did  a little sanity check for my specific sensor.

Now that I can get data from the sensors time to work on getting the data online...

Thursday, March 13, 2014

RF Propagation Modeling & a Field Test


One of the biggest risks of this project hinges on the fact that the remote weather station needs to be able to connect back to my base station, or it has no chance of success. Given that, I decided to undertake some RF Propagation Modeling to boost my confidence in the final system as well as determine such things as antenna placement (elevation angle/direction) and antenna type.

Due to the miracles of the internet (and procrastination), the blog is slightly behind the reality of where my project is, so I captured below the RF Modeling that took place (spoiler alert) and the field test to check the RF Modeling. As you might guess, the field test only only took place since there was at least a  glimmer of hope (Read: ~10dbm wiggle room) that the radios would connect.

After getting the DNT900 and RPi's working together nicely, it seemed a prudent time to do some more research before investing in all the other ancillary hardware to pull the remote weather station off.  Having some notion of the concept of RF Propagation Modeling, I began to search online for some tools that might help with this. (Perhaps if I had been a classically trained engineer I could have determined my link budget with pen and paper, but that is not my case, and I generally like pretty pictures and well made software.)

A search online yielded an helpful presentation(PDF) from John Hopkins ARL on Modeling and Simulation for RF Propagation. For those looking for a technical review on the subject, check it out, good read.  I happened to also use it for some ideas for possible tools to do the modeling.  Page 16 of the presentation has all sorts of suggestions. My initial criteria was simple: free (or free during trial) & decent usability (or good reference materials). Luckily I didn't have to explore for too long before finding a great solution: SPLAT!  The balance of this post includes output from SPLAT! & Google Earth.

I will also mention I tried  a software package called TAP (Softwright Terrain Analysis Package) but found it too difficult to ramp-up on (perhaps it is meant for the a very patient and forgiving  professional), I only gave it an hour of my time after install before bailing with nothing to show for it.) Perhaps not fair to TAP, but I was looking for a fairly simple answer/analysis, and the work to get that from the tool was too painful.

Some inputs to SPLAT! Included knowing the location of my antennas, EIRP/ERP, local terrain data (SPLAT! provides instructions to import); also referenced dB/Watt conversion  and Free Space Loss for sanity checks.

Going in I saw the DNT900P data sheet made claims about 40+  Line-of-Sight (LOS) coverage, an item that I needed to check for my setup. The data sheet also contains helpful nuggets about receiver sensitivities (the later being something closely looked at during this analysis). I first used Google Maps to determine the Lat/Long of my Base and Remote antennas. I also found this Elevation Profile view helpful to casually observe possible obstacles and get a feel for the elevation that would play into my ability to (not) achieve LOS.


So in summary so far:
Base: Elevation 289ft  (actually ~ +40 in attic  ) ~329
Remote: Elevation:   328 (if I can get +10 feet off the ground)
Distance 1.64 Miles

Using Google Maps Ruler tools I also was able to find the heading needed for each Antenna. (Nice to have in hand & to check against SPLAT! values, which concurred.)


Base to remote: ~192 degrees.
Remote to base: ~12 degrees



Getting going onto SPLAT! just required a few files, of which I provide generic examples of here. The actual SPLAT! site details these files in greater detail but feel free experiment with these if it can get you using the tool any quicker. I chose to Model Omni Directional antennas for simplicity.Here are some of the SPLAT! outputs.

Below we see the anticipated receive dBm of the remote, as related to the base:


Below we see the anticipated receive dBm of the base, as related to the remote:


Pictures are valuable, but SPLAT! also provides a detailed report, here is a sample from a similar model to shown above.

From the prior data I get a feel for the impact of not achieving LOS and anticipated receiver strengths for the sights, all just barely within spec of DNT900 which called for -98 at a 200 kb/s data rate. The input values could be tweaked to achieve the desired result (aka adjusting elevation and antenna gain where necessary).

SPLAT! also puts out some data in a convenient KML format which actually does a nice job visualizing the LOS obstacles. (Base to remote LOS obstacles, followed by remote to base below.)








At this point, I consulted a few co-workers who encouraged me that lack of LOS was not a show-stopper, but given the narrow margin of theoretical receive dBm (on both ends), as compared to DNT900 radio sensitivity it was a sketchy area (that could be impacted/fail on loss from poor cabling/connector choices, vegetation, precipitation, or un-describable obstructions*.) The model takes most of that into account, but it is a difficult to know how accurate is compared to the real world.  *Due to HOA rules my home-base antenna location is less than ideal. It is inside my attic, so I have the unknown variable of plywood and tar shingles in the mix. I think I ball-parked  an additional -20dBm for this in my model.

Being 'forever the optimist', I felt moderately confident to purchase my antennas and get ready for a field test. Installing my home-base station (RPi + RPi900 + DNT900) was abit cumbersome in the attic, requiring power and network to be run. Network simplified by using an old SlingNet Ethernet-over-power device. I wasn't too worried about the bandwidth limitations of such an install, as my bottle neck for data is really between my remote and base station.

With my base station in place, I prepped my remote station for a field trip to the garden. The setup was crude, but effective. I mounted the antenna, battery-pack and  RPi remote system (RPi + RPi900 + DNT900) to a good 'ol Hula Hoa (which from a gardening perspective, I also highly recommend.) The remote station-on-a-stick setup was to facilitate testing different locations and elevations within the garden.


As I ventured off to the garden, I had my wife monitoring a computer at home logged into my base station. I already configured my RPi900's to establish a PPP network, so she was monitoring a ping I had going to the remote unit I carried. I was keeping an eye on the RPi900 indicator lights to know when to ask her to pay closer attention. Here is an one of the first examples of the system establishing a PPP network connection, over the 1.64 mile RF connection:

From basepi (192.168.1.###) icmp_seq=1988 Destination Host Unreachable...........
64 bytes from remotepi (192.168.1.###): icmp_seq=2008 ttl=64 time=49.3 ms
64 bytes from remotepi (192.168.1.###): icmp_seq=2010 ttl=64 time=60.4 ms


When confident the DNT900 radio's were linked (steady green light) the dnt900 line discipline was used to check the receive rssi (signal strength) at the base, of the remote RPi I had in the field (garden).

[@basepi ~]# cat /sys/class/dnt900/ttyAMA0/0x00####/rssi
-88
[@basepi ~]# cat /sys/class/dnt900/ttyAMA0/0x00####/rssi
-91


A shot of the field test setup above when working (perched about 10 feet above my head) on a convenient pole on the perimeter of the garden.  From the field test I learned the very center of the garden was pretty much an RF nightmare. All sorts of metal caging around the individual plots, and the large deer fence around the parameter of the garden acting like a large Faraday cage (I believed.) My success was on the perimeter of the garden - getting in-front of, and above, those immediate impediments. (The SPLAT! modeling did in-fact predict the need for greater antenna height at the remote sight then I orginally planned.)

I am going to claim success on this field test, and endorse SPLAT! since it was extremely close in it's prediction (of between -80 to -90 dBm receive strength), and pretty easy to use.  I was able to find a few potential locations for the remote weather station at the garden, all establishing network connections at the 2nd lowest data rate (with approximately 7 dBm to spare.)

I also still have the option to drop down to an even lower data rate (38.4 kb/s and gain an additional 10 dBm of operability), since the radios are sensitive to -108 dBm, which seems believable given the performance at 200 kb/s.  I think I can also eek out a few dBm in both RPi900/antenna setups by ordering high-quality (lower loss) cable, and limiting some adapters in the mix.

 I didn't  explore the DNT900 claims of 40+ miles of LOS operation from a real world test, or even bother modeling it. I'd imagine it could be possible  in perfect/ideal conditions, or maybe in airborne/ground station setups. Either way, I'd suggest everyone should model their respective communications TX/RX setups (even on an existing functional system), I found it a helpful and informative exercise.