Sharp Wireless Weather Station, part 3
Reverse engineering the CRC
In looking at the crc field, I tried a number of different things.
rtl_433 crc functions
First, I tried the crc7, crc8, and crc8le functions from rtl_433. The polynomial and init parameters are in the range of 0..255, so I just tried a brute force of every polynomial+init parameter combination against a small list of messages. None of the tests turned up valid parameters that worked for every message, so I put those functions aside.
Hamming codes
Next, I tried Hamming codes. These are nice because they provide correction for a single bit error, and (depending on the redundancy used) detection of a double bit error. The messages didn't fit any of the standard Hamming code patterns, so I started looking at reverse engineering.
With a Hamming code, one bit change can cause multiple bits to flip in the parity check bits. It uses xor combination instead of addition, so a change in a bit in the data always changes the crc bits in the same way no matter what the other bits of the data are set to. Knowing this, I started by comparing messages that had 1 bit of difference in the data from each other.
For example:
Between these two messages, only bit #32 is different in the data. This is the least significant bit of temperature, representing 0.1 degree C. With this data difference, three bits of the crc flipped as well - #43, #44, and #48. So we can conclude that bit #32 affects those fields. I can look at all other pairs of messages with just bit #32 flipped and see if they have the same crc bits flipped as well. They did, so this looked to be a good path to continue on.
After finding the one bit differences, I could look for two bit differences where one of the bits was already known. After adjusting the expected crc for that already known bit, I could use the remaining difference between expected and actual to find that data bit's contribution to the crc bits. After those are done, I can do three bits changed with two known bits, four bits changed with three known bits, etc. I took a few days worth of data and built up a table of these differences.
To check a message, I filled in the "changed" column as well as the "actual crc" row. The "crc 4x" columns are 0 or 1 based on the changed column on their row. They are added up, modulo 2, to complete the "expected crc" row. Then the "expected crc" row can be compared to the "actual crc" row.
This isn't all the possible data bits of the messages, because only some of the data bits change in any given 24 hour period. But there is a pattern here. The same bits are flipped for #32 and for #40, the same for #31 and #39, and the same for #30 and #38.
Because the same pattern is applied to multiple different data bits, this can't be called a Hamming code. But it will do fine for a simple message valid check.
Applying the assumed crc function
Starting with the assumption that all data bytes had the same crc patterns based on the Bit # in Byte, I tried applying it to a whole message:
I filled in the "bit value" column as well as the "actual crc" row, and the actual crc and expected crc matched!
I applied this to all the messages I've received so far and got: 61,819 good crc and 50 bad crc. I assume those 50 were from decode errors due to rf noise. This is an error rate of 0.08%!
I am satisfied I have the crc message fully decoded, so let's move on to some data.
Graphs: Temperature and Humidity
This is the temperature between July 29 and August 5.
And the Relative humidity for the same timeframe. From the two graphs, you can see that when the temperature goes down, the relative humidity goes up. A simplified explanation for this is because the amount of evaporated water that air can carry depends on its temperature.
Graph: Messages
Looking at the time between messages, 66% were spaced between 56.90s and 56.99s, 33% were spaced between 56.82s and 56.89s, and all of them were spaced between 56.82s and 57.06s. Right now, I have the receiver running all the time. But one optimization I could make is to turn the receiver off between messages. I would need to allow for time for the receiver to start back up, and the receiver would need a way to handle losing signal completely.
Lost messages
Speaking of lost messages, if I expect 3 messages every 56.93 seconds, I should have 19,736 groups of 3 messages in 13 days of collected data (exactly 1,123,580 seconds). I actually received 19,706 groups of varying numbers of messages (0.2% of groups missed).
In those groups, I should have seen 59,208 messages. I actually received 56,990 messages, a 3.7% loss. The message being repeated 3 times helps reduce the impact of that loss rate.
Future parts
- Turn the receiver off between messages
- Automate the temperature/humidity graphs
- Include a measurement of signal strength and frequency
- Adding this model to the rtl_433 project