Hi, I am Liudr, a Tesla member at the arduino forum. This blog is an extension of my activities on arduino forum and my raspberry pi activities. Get latest updates through this blogs about the projects done using Arduino and sensors.
Now that the logger prototype is enclosed, it is a lot easier to use. I have a METER group 5TM soil sensor connected to it from the left side and 12V power to it from the bottom. Without a heavy enclosure, these cables will make it hard to set the logger flat and connect an adapter to it. Plus, I’ve started using Telnet to connect to it wirelessly so I don’t need an adapter to connect to it most of the time.
Here is the adapter I modified to use on the logger:
I bought it from moderndevices.com (Not shown on this photo) Later I had to short a resistor and a capacitor in order to actually make it work, due to the fact that the adapter was designed for Arduino boards.
Besides Telnet, I can also upload Python sketches via FTP. I later installed a couple of jumpers to easily reset the logger since I am still developing the logging script and need to reset the logger when I make changes. I’ll install a push button instead of the jumpers when I have some time.
Everything is coming along nicely. Soon I’ll start testing 4G LTE-M connectivity of the logger. 4G LTE-M has a lower bandwidth than the regular 4G LTE on smartphones so the modem would consume less power and data plans would cost less (in Megabytes instead of in Gigabytes). The Digi XBee3 LTE-M modem is rather expensive, at $100, or $150 for a dev kit. Anyway, if you are deploying the logger in a research field close to your lab/office or really don’t mind regularly visiting remote areas to collect data logs, you may skip the modem.
I recently worked on a project that required a magnetic sensor (MPU-9250) be calibrated to get best accuracy. I had some basic understanding on how to calibrate a magnetic sensor, although I’ve not done calibration before. The calibration turned out to be a bit complicated and took a while to understand. I got some help, including method to get best calibration, from the author of the MPU9250 Arduino library, Kris Winer. I thought that if I shared my experience here, others that are planning to calibrate their magnetic sensors may find it useful.
First of all, what is calibration? In a general sense, calibrating a sensor makes the sensor provide the most accurate readings allowed by the sensor’s own precision. As an example, let’s assume for a moment that the earth’s magnetic field and any other stray magnetic fields are shielded and you have a uniform magnetic field generated artificially for the sole purpose of calibration. Let’s say that the field strength is 400 mG (milliGauss), equivalent to 40,000 nT (nanoTesla). Now if you align one axis of your magnetic sensor parallel to the direction of the field, it should read 400mG. If you then carefully rotate your sensor so that the axis is anti-parallel with your field, it will read -400mG. If you didn’t do a good job in either alignments, you will read less values, say 390mG, if you’re off by about 13 degrees, because only a portion of the field, which is a vector, is projected along your magnetic sensor’s axis.
In the diagram above, the thick blue arrows represent the constant magnetic field of 400mG pointing to the right. The thin arrows represent various orientations your sensor could take. If your sensor + axis is also pointing to the right, you get the full 400mG. If your sensor + axis points to the left, you get -400mG. If your sensor + axis makes an angle, it reads a projection of the field, which is less. You can figure out the angle:
The above was assuming that there IS a constant magnetic field and the sensor’s reading IS symmetric along its positive and negative axis, meaning with zero magnetic field, the sensor reads zero. When not calibrated, the sensor reading will NOT be zero under zero field. It could read say 10mG. As a result, you might get say 510mG and -490mG with the field on. You know what that means. There is an offset (bias) of 10mG that should be subtracted from your reading to get the correct reading of +-400mG.
The above was the basis for calibration to remove the offset (bias) on each axis. In order to get the maximal and minimal value, you need to write your code to store max/min out of a stream of live data while you rotate your sensor in space, trying to maximize or minimize the readout. Then repeat two more times for a 3-D magnetic sensor. Since you don’t have a magnetic shield, you are relying on the earth’s magnetic field as the constant field. The earth’s magnetic field is not horizontal, or pointing from north to south. In most areas, the field either has a vertical up component, or a down component. And in most cases the field points from south to north as the rotational north pole (AKA the north pole) is near the magnetic south pole, where field lines go in, not coming out. In my area for example, the earth magnetic field points primarily downwards, only slightly towards north, making an angle over 70 degrees with the horizontal. The magnetic field has very little component in the east direction. The relative strength between East, North, and Downward is about 1:64:195. The angle the magnetic field vector makes with the horizontal is called magnetic inclination, with downward being positive. This is approximately atan(195/64)=72 degrees. The angle the magnetic field vector’s horizontal component makes with the true north is called magnetic declination, with east being positive. This is approximately atan(1/64)=0.9 degrees. The properties of magnetic field varies greatly from place to place and also changes from time to time. To find out the magnetic field in your area, visit noaa.gov:
The next calibration is for sensitivity. The sensor either returns an analog voltage or a digital value. How do we convert this return into actual magnetic field in mG? This means finding the relation between the sensor readout and actual physical values. Say the sensor is digital and returns values between 0 and 32767, which represents magnetic field between 0 and 49150mG. Then you can use the conversion vactor 49150/32767=1.5mG/LSB to convert your readout. Here LSB means one digit (least significant bit). For an analog sensor, you will need an x.xx mG/V.
All sensors provide this factor in their spec sheets so you can just use this factor to get the actual magnetic field. But since not all sensors were made identical, some sensors should use larger or smaller values than the spec’s factor. Some manufacturers test their sensors at factory and store a correction factor for each axis in the sensor for better accuracy. For example, the MPU-9250 sensor (the magnetic sensor is AK8963) has digital magnetic field sensor output. One of the sensors I got has the following factory trims:
X-Axis sensitivity 1.18
Y-Axis sensitivity 1.19
Z-Axis sensitivity 1.14
So instead of a straight 1.5mG/LSB, the x-axis has 1.5*1.18=1.77mG/LSB. We’ll multiply this factor to the x-axis readout to get x magnetic field in mG. Same for y and z axes.
And yet sometimes these adjustments are still not able to make all 3 axes read the “400mG” value. They are very close to be identical already so we can apply a small correction. We average the maximal readings from all three axes, then divide by the maximal reading of each individual axis to get three factors. If say the x-axis reads slightly higher maximum than y and z axes. Then the avg_max/x_max will be slightly less than 1. We apply this factor to the the final result:
For one of my sensors, this yields 1.5*1.180*0.975*x_readout=1.726*x_readout(mG)
If the sensor you’re using doesn’t have the factory trim, then you’re out of luck unless you have both a magnetic shield and a nice uniform magnetic field inside the shield so you can find out the trim.
Extract factory sensitivity factor by calling initAK8963().
Call readMagData() to extract raw data repeatedly enough to get accurate max/min for all axes.
Calculate bias in raw counts by (max+min)/2, such as (510+ (-490))/2=10
Combine factory sensitivity factor and Kris’s scale factor described above.
Keep these biases and factors in program.
As a testimony to the method that works, here are two graphs.
This graph has three separate plots from raw data that represent mx vs. my, mx vs. mz, and my vs. mz. I rotated my sensor as much as I could for a few minutes before I got bored and couldn’t think of any other ways of rotation. The fact that all three plots are near circular means that the sensor’s three axes have very similar sensitivities. I was simply rotating the sensor around, giving all three axes opportunities to read the whole magnetic field. This means when the x axis reads the whole field, y and z axes read nothing. Some trigonometry can show that the result is a series or circles making a disc. But the discs were not centered as I expected, because the z axis had a very large bias (this means the blue is mx vs. my). After my bias calibration, here is what I got:
Now all three plots are centered at zero pretty well visually, although I didn’t multiple the factory sensitivity or the final factor from Kris’s calculation. This shows that bias calibration is the most crucial. If your sensor doesn’t have stored factory sensitivity factors, getting it calibrated for bias alone will go a long way.
FYI, this is my sensor board inside an enclosure. I found it much easier to hold and rotate when it’s in a box. The cable became much less of an issue when I was rotating the box. The black square board is my SDI-12 USB adapter. The purple board is the sensor board. I customized the SDI-12 USB adapter by gluing the sensor board to it and connecting it to the I2C bus on the adapter.
I have been working on some updates to the SDI-12 USB adapter so that it would add more features to a data logging system. So far, I’ve updated the PCB (left board) to include additional connectors. The top of the board will have 4 analog channels. This is not as accurate as the red SDI-12 + Analog USB adapter boards. The SDI-12 + Analog USB adapter has practical accuracy of 20 microvolts and has differential input channels. The 4 channels on the basic SDI-12 USB adapter have accuracy of about 5 millivolts. Also there is not a voltage reference so the measurement will be affected by the USB voltage, which is only nominally 5V. Nevertheless, if there are some sensors that output voltages in 0-5V range you want to log with moderate accuracy, such as a potentiometer, or a thermistor for approximate temperature calculation, you can use these channels. The breakout looks the same as the SDI-12 + Analog USB adapter. There are no serial resistors so you have to add yours if you want to convert resistance to voltage.
I will release a new firmware version on these newer boards. At the same time, I am considering adding digital counting features to these analog channels so if someone wants to count pulses such as flow meters or rain gauges, they can use these channels for such purpose. I plan to develop this part in the summer.
Another connector (bottom one on left board) I have added will connect the adapter to an analog extension board (right board), which sports the same 20 microvolt accuracy as the SDI-12 + Analog USB adapter, in case you want to add these channels say for pyronometers or other low voltage and high precision measurements after initially getting the basic adapter. You can stack up to 4 such analog extension boards. Each board has an address jumper (right board, white rectangle) for one of the four addresses the analog-to-digital converter supports. That gives you a total of 16 high-precision analog input channels. Each extension board also comes with a few additional connectors for SDI-12 sensors as an option. You can more easily wire up more SDI-12 sensors to the adapter. I don’t recommend wiring up more than 6-8 SDI-12 sensors to the same adapter. Some sensors come with strong pull-down resistors. When too many of them are wired to the same adapter, they may prevent some other sensors from correctly communicating on the bus.
The last connector (left board middle) I have added will help developers using MicroPython platforms easily connect to it via serial ports, since most MicroPython boards don’t have USB hosts. I will start shipping these newer SDI-12 USB adapter boards soon although new firmware that makes use of these features will have to wait until later.
Will this affect your existing projects? Very unlikely. The new adapter has all the features of the old adapter. The SDI-12 + Analog adapter will still be around since it is a nice compact form factor. The new SDI-12 adapter plus the analog extension board will be approximately the same price as as the SDI-12 + Analog adapter.
If you have heard of the Python programming language and its easy-to-learn and easy-to-use fyou’ll be pleasantly surprised that someone has successfully implemented Python on microcontrollers a few years ago, appropriately named MicroPython! He has developed his own MicroPython board and ported the code to a number of similar microcontrollers. Imagine a low-power microcontroller with “lots” of memory (compared with Arduino) happily running Python code that talks to the internet etc. I’ll start writing about MicroPython and how you may use it for DIY electronics and data logging in a number of posts but this post is an announcement related to MicroPython and the SDI-12 serial dongle:
This dongle has a similar set of features as the SDI-12 USB adapters but lacks the USB connectivity, just having serial connection to arduino. Since all MicroPython compatible boards have serial ports and library to run the port, you can connect a MicroPython compatible board to one of these dongles and run almost the same Python data logging code I provide to SDI-12 USB adapters. Here is the first successful attempt that made it happen. Jason is a developer on the Wipy platform, which is a microcontroller supporting MicroPython. He is developing a data logger. In order to talk to SDI-12 sensors, he got one of my SDI-12 dongles. The Wipy board has serial ports but at 3.3V logic. The dongle has 5V logic. I built a simple voltage divider on the dongle for him and he was able to communicate with it using the Wipy board. Here is the code that he wishes to share:
Due to the discontinuation of data.sparkfun.com, I moved data logging to ThingSpeak.com
I have other updates that I rolled up in the manual, such as more details on telemetry. New manual is posted on SDI-12 USB adapter page as well as the updated data logging code. Here is a snapshot of data I logged to ThingSpeak.com: