What the results say…
We’re confident that the machine learning model has the potential to help create a more consistent interpretation of test strip colours, but further calibration is required to improve its accuracy!
A study to help us make colour tests more accurate!
In 2019, with the help of participants across Canada, we conducted a study which helped us create reference photos to better read inexpensive colour-based water tests. This first step created references for two types of test strips (high-level phosphates and alkalinity, pH, and hardness).
We sent participants our ‘testers’ package, which included test strips, a reference card, field sheets, and reference materials. Each test took less than 5 minutes to conduct. Participants included NGOs (like SwimDrinkFish), Conservation Authorities, scientists, and regular folks from across Canada!
Why this project?
Many inexpensive water tests are analyzed through colour comparison, but eyes interpret hues differently (for example, pH strips can have wildly different results based on their readings from orange to pink, which is affected by common types of colour blindness). Color-sensing technology is often inaccurate because lighting affects sampling.
We are looking at how machine learning applies an algorithms that ‘read’ colours, and, through integrating different datasets, helps create meaning for citizen science water testers.
This project was designed to help us verify an artificial intelligence model designed to adjust the image and detect the true color and value of the test strip. This allows for the elimination of human error and for data to be accurate: the model could interpret a finer scale not detectable by the human eye.
We also explored how to legitimize crowd-sourced data so it can be used by academics and policymakers in decision-making and consulting academic partners to understand rigorous protocols.
This method and machine learning model can be extended to read more tests (we are hoping to soon ‘read’ the liquid test for dissolved oxygen, for example). If you know any other tests that you think might be useful for our testkits, please email firstname.lastname@example.org.
How the study worked
Eventually, we want phosphate strips as an ‘early detection’ for high levels of phosphates in water.
We suggested that tests be taken in water suspected to be contaminated by a sewage overflow, agricultural run-off, or faulty septic systems at the source, where you expect phosphate readings of 0.1ppm (100ppb) or more.
Older lakes and ponds were deemed to be suitable candidates for this study. Healthy, young lakes were not. This is because regular lake testing in Ontario would be around 0.031ppm (10ppb of phosphorus) and so wouldn’t be suitable for these tests (NOTE: To convert phosphates to phosphorous, multiply the ppb value for phosphorus by 3.066 then divide by 1000 to equal ppm phosphate).
So how did it go?
After conducting this study, we’re confident that the machine learning model has the potential to help create a more consistent interpretation of test strip colours. From correspondence with testers, we found that citizen monitors are excited to use their phones rather than expensive equipment to interpret results, and have said they are happy that it will increase the accuracy of their data so that it can potentially be used in decision-making.
So why isn’t this feature available on your app?
Firstly, further calibration is required to improve the accuracy of the studied system, and more data will make the model increasingly accurate. Secondly, phosphorus testing is difficult! Currently, there aren’t any test strips out there that are a.) sensitive enough to detect the levels of phosphorus we see in many lakes in Canada, and b.) live up to the Water Rangers standard, in that they are safe, affordable, accurate, easy to understand, and easy to use. Many tests are in parts per million, but in order to give us an accurate idea of what’s actually going, they would need to be in parts per billion (aka, one thousand times more sensitive than ppm). There are some technologies out there that claim to do this, but that don’t meet the Water Rangers standard.
Don’t worry though- we’re still working on making the use of this technology a reality! Right now, we’re looking at how this technology can be used to interpret pH test strips, and we have people testing out this feature on their phones already. More information about this will be available in early 2021. Want to know more? Contact us!
This project was funded by CIRA’s Community Investment Program, who are supporting us to build a stronger Canada online. Thank you for supporting us!