PalayCheck
A rice disease scanner that would rather say “not sure” than guess.
Farmers photograph a rice leaf and the phone checks the photo, checks that it is actually a rice leaf, and only then runs the disease model. Results the model isn’t sure about go to the Municipal Agriculture Office instead of showing a confident wrong answer.
- Lead developer (app, backend, ML pipeline)
- Team capstone
- Jul – Oct 2026
- Android · officer dashboard
- Flutter, TensorFlow, TFLite (INT8), Python, Supabase, flutter_map
- Not yet released · pilot scope: Aparri MAO
- Private repository
Capture
Camera or gallery photo, orientation fixed from EXIF.
decoded in an isolate
Quality check
Too dark, too bright or too blurry to judge?
mean luma 45–215 · Laplacian variance ≥ 60
If not: Retake, with the reason
Rice-leaf gate
Is this a rice leaf at all?
MobileNetV2 α0.5 · INT8 · 0.97 MB
If not: Not a rice leaf
Classify
Crop to the leaf (+10%), then six classes.
MobileNetV2 · INT8 · 2.7 MB
Confidence gate
Sure enough to name a disease?
top ≥ 0.45 · margin ≥ 0.10 · entropy ≤ 0.65
If not: “Not sure”: report to the MAO
Result
Fuse several photos into one result.
rejected photos never leave the phone
Introduction
PalayCheck helps rice farmers check a leaf for disease with their phone, and gives the Municipal Agriculture Office (MAO) in Aparri a map of where problems are being reported. It was our team’s capstone project, and I led development: the Flutter app, the Supabase backend and the machine-learning pipeline.
It isn’t released. The landing page (opens in a new tab) is live, the pilot scope is the Aparri MAO, and a formal acceptance run on the target phones is still to come. This case study is mostly about one question: what should a disease scanner do when it isn’t sure?
Problem and motivation
A disease scanner can fail in two ways: it can miss a disease, or it can invent one. For a farmer deciding whether to spray, both cost money, and both are worse than an honest “I don’t know.”
Image classifiers make that hard, because they always have an answer. A six-way softmax sums to 1, so a photo of a hand, a blurred leaf or a weed still gets a “top” label with a number next to it. The model can’t tell you it’s out of its depth, so the app has to.
The second problem showed up later: a model can look excellent on its own test set and much worse on photos from somewhere else.
Users and context
Two groups use PalayCheck, in the same app:
- Farmers take photos in the field, often in bright sun, on low- to mid-range Android phones with patchy data. The app works in English and Filipino, and scanning works without a connection.
- Agriculture officers at the MAO sign in to their own set of screens to review reports, see where they cluster on a map, and send decisions and recommendations back.
The phones matter. The compatibility plan targets devices like the TECNO CN7c and phones with 2 GB of memory. That rules out large models, and it makes the GPU an unreliable partner, for reasons covered below.
Role and contributions
PalayCheck was a team capstone and I was the lead developer. I built the Flutter app, the Supabase backend and the ML pipeline: assembling the datasets, the training scripts, the INT8 export, and the evaluation that decided which model shipped.
As on my other projects, I worked with AI coding assistants as pair programmers. The design, the decisions they were used for, and the review of what they wrote were mine.
Technologies
- Flutter and Dart · English and Filipino
- TensorFlow Lite, INT8 · inference in a background isolate · CPU (XNNPACK) only
- MobileNetV2 α1.0 disease classifier, 6 classes, 2.7 MB · MobileNetV2 α0.5 rice-leaf gate, 0.97 MB
- Python and TensorFlow · tensors produced by the app’s own Dart decoder
- Supabase · Postgres with row-level security · pgTAP
- flutter_map with OpenStreetMap tiles · hotspots computed in SQL
- 45 Flutter test files · a pgTAP suite
System architecture
The scan pipeline at the top of this page runs entirely on the phone. Photos that fail the quality check or the leaf gate never leave it. Scans and reports go up through an outbox when there’s a connection, and officers see them on the other side.
Scan pipeline
quality → leaf gate → classifier → confidence
Multi-photo fusion
one diagnosis from several photos
Outbox
scans and reports, ids made on the phone
Farmer’s phone to Supabase: scans and reports, upserted by id
Supabase to Farmer’s phone: decisions and recommendations
Postgres + RLS
scans, reports, report events
hotspots()
SQL · last 14 days, at least 2 reports
Push notifications
decisions and new hotspots
Supabase to Officer screens: reports and hotspots
Officer screens to Supabase: decisions
Outbreak map
flutter_map + OpenStreetMap
Reports
review, decide, recommend
Analytics
Every scan and report gets its id on the phone, and the upload is an upsert by that id, so a retry after a dropped connection can’t create a duplicate. Reports also record where their location came from, so an officer can tell a precise fix from a rough one.
Key implementations
Three questions before an answer
Before the classifier runs, a photo has to pass a quality check (brightness and sharpness) and a separate rice-leaf gate: a smaller model trained only to answer “is this a rice leaf?” After the classifier runs, its output has to pass a confidence gate. Any one of three signals failing makes the result “not sure”:
/// The classifier's highest probability is not automatically an answer: a
/// 6-way softmax always sums to 1, so even a photo that matches no class
/// well produces a "top" label.
PredictionDecision evaluate(List<double> probabilities) {
var top = 0.0;
var second = 0.0;
for (final p in probabilities) {
if (p > top) {
second = top;
top = p;
} else if (p > second) {
second = p;
}
}
// Quantized outputs don't sum to exactly 1; normalize before entropy.
final sum = probabilities.fold<double>(0, (a, b) => a + b);
var entropy = 0.0;
if (sum > 0) {
for (final p in probabilities) {
final q = p / sum;
if (q > 0) entropy -= q * math.log(q);
}
}
final normEntropy = probabilities.length > 1 ? entropy / math.log(probabilities.length) : 0.0;
final margin = top - second;
final reasons = <UncertaintyReason>{
if (top < thresholds.minTopProbability) UncertaintyReason.lowTopProbability,
if (margin < thresholds.minMargin) UncertaintyReason.smallMargin,
if (normEntropy > thresholds.maxNormalizedEntropy) UncertaintyReason.highEntropy,
};
return PredictionDecision(
verdict: reasons.isEmpty ? PredictionVerdict.accepted : PredictionVerdict.uncertain,
reasons: reasons,
topProbability: top,
margin: margin,
normalizedEntropy: normEntropy,
);
}The thresholds weren’t picked by feel. A top probability of at least 0.45, a margin of at least 0.10 over the runner-up and a normalized entropy of at most 0.65 were chosen on the INT8 model’s validation outputs, as the widest coverage that keeps errors among shown results at 5% or less. On validation, 96.8% of results are shown and 4.9% of those are wrong. A test fails if the app’s thresholds ever drift from the model’s metadata.
Training on the bytes the phone sees
Training pipelines usually decode images with Python libraries; the phone decodes them with Dart. I checked whether that matters. Resizing matched exactly, but JPEG decoding didn’t: Python’s decoder and the Dart one differ by 0.6–3.4 levels on average, and by up to 57 on single pixels.
So every training, validation and test tensor is produced by the app’s own decode and resize code, run as a Dart test, and the model learns from exactly what the phone will feed it. After export, a parity check runs the app’s runtime and Python on the same 60 photos: they agree on the top label for all 60, with a mean probability difference of 0.009.
Several photos, one diagnosis
A farmer can take more than one photo of a leaf, and the results are fused into one diagnosis. Disagreement has its own rules. If one photo points to a disease while another confidently looks healthy, one misread photo shouldn’t become the whole scan’s answer, so the result is “unclear” and names the disputed disease.
Challenges
The test set was too kind
On its held-out test set, the disease model scores 93.3%. On 382 photos from a separate field dataset, the same model with the same weights scores 75.7%.
- Held-out test set
- 93.3%1,894 photos from the training sources, never trained on
- Separate field dataset
- 75.7%382 photos, closer to what a phone in a rice field sees
The model before it, v0, showed the same pattern more sharply: 93% brown spot recall on its own test set, 45% on field photos. Worse, it called diseased leaves “Healthy” with high confidence. 28 of its 91 “Healthy” verdicts on field photos were wrong, 20 of them brown spot.
A gate that turned away good leaves
The rice-leaf gate is there to stop photos of anything else. On field photos it was also rejecting 42% of the rice photos that had passed the quality check, including 72% of the sheath blight photos, so one disease was nearly invisible. Its threshold was re-chosen on validation scores before any field data was used to judge it.
Lesions a few pixels wide
Rice blast and brown spot both appear as small lesions, and at the model’s 224-pixel input they’re only a few pixels across. The shipped model still confuses them: in the field benchmark, blast was called brown spot 14 times and sheath blight 10 times. The likely fixes, a higher input resolution or tiled inference, change the app’s model contract, so they’re next rather than done.
Decisions and trade-offs
CPU only, on every phone
- TensorFlow Lite on the CPU, through XNNPACK.
- GPU or NNAPI acceleration.
- GPU and NNAPI numerics vary by vendor. The same photo should get the same diagnosis on any phone.
- Slower inference on phones with a capable GPU, so inference runs in a background isolate to keep the interface responsive.
Ship v1, despite two missed criteria
- The v1 model, with its two misses recorded in the model’s metadata under shipped_despite.
- Keeping v0, which the pre-registered rule says to do when no candidate passes everything, or shipping v2.
- v1 was the only candidate to meet the brown spot target, it cut diseased leaves called “Healthy” from 28 of 91 to 3 of 79, and it beat v0 overall.
- Weaker rice blast recall, down from 0.432 to 0.273, and slightly lower sheath blight recall. Because the rule said don’t swap, the decision went to the project owner, who accepted both trade-offs in writing.
The five criteria were written down before the benchmark ran. Here is how the shipped model did against them, on the 336 field photos that pass the quality check and the leaf gate:
| Measure | v0 | v1 (shipped) | Pre-registered target | Result |
|---|---|---|---|---|
| Accuracy | 61.9% | 74.1% | – | No target |
| Macro-F1 | 0.599 | 0.714 | ≥ 0.67 | Met |
| Brown spot recall | 0.446 | 0.714 | ≥ 0.70 | Met |
| Diseased leaves called “Healthy” | 28 / 91 | 3 / 79 | Fewer than v0 | Met |
| Rice blast recall | 0.432 | 0.273 | No more than 5 points below v0 | Missed |
| Sheath blight recall / precision | 0.887 / 0.522 | 0.849 / 0.549 | ≥ 0.85 / ≥ 0.70 | Missed |
Say “not sure” instead of guessing
- Hold back results that fail the confidence gate, and offer to send them to the MAO.
- Always showing the top label with its confidence.
- A confident wrong answer costs a farmer more than an honest “not sure”.
- Some correct answers are held back too: on validation, 3.2% of results aren’t shown.
Results and validation
These numbers come from public datasets, not from farmers’ phones. The model’s own experiment record says so directly: none of them come from PalayCheck users’ phones in Philippine fields. The benchmark also shares source datasets with training. Near-duplicates were removed, but photographers and styles overlap.
| Class | Test | Field | Chart |
|---|---|---|---|
| Bacterial leaf blight | 96% | 53% | |
| Brown spot | 92% | 69% | |
| Healthy | 96% | 98% | |
| Rice blast | 90% | 39% | |
| Sheath blight | 96% | 96% | |
| Tungro | 93% | 100% |
| BLB | Brown spot | Healthy | Blast | Sheath | Tungro | |
|---|---|---|---|---|---|---|
| BLB | 39 | 4 | 0 | 2 | 27 | 1 |
| Brown spot | 8 | 44 | 3 | 5 | 1 | 3 |
| Healthy | 0 | 1 | 78 | 1 | 0 | 0 |
| Blast | 8 | 15 | 1 | 22 | 10 | 1 |
| Sheath | 2 | 0 | 0 | 0 | 51 | 0 |
| Tungro | 0 | 0 | 0 | 0 | 0 | 55 |
- Through the whole on-device path, accuracy on those 336 field photos went from 61.9% with v0 to 74.1% with v1.
- The app’s runtime agrees with the Python evaluation on the top label for 60 of 60 photos.
- An integration test pins the bundled model’s bytes, metadata, labels and thresholds, so a model can’t change silently.
Not done yet:
- the formal acceptance run on the target phones;
- the field protocol for the rice-leaf gate, which stays marked as not validated until it passes;
- calibrating the photo-quality thresholds, which the code still marks as provisional;
- photos from farmers’ own phones in Philippine fields, which is the data the next model needs most.
A known gap: one queued action, an officer’s decision on a report, isn’t fully idempotent yet. A retry after a call that committed but timed out could notify the farmer twice. The app checks the report first, which narrows the window without closing it. The complete fix needs a migration, and the code says so.
Lessons learned
- A test set result is a claim about that test set. 93.3% was true, and it still said little about the field.
- Decide the criteria before you see the results. Pre-registered targets turned the v1 decision into a visible trade-off instead of a story told afterwards.
- Train on what the device sees. A decoder difference too small to notice by eye was big enough to measure.
- Design the “no” first. A lot of what makes the app trustworthy is the paths that end in “retake”, “not a rice leaf” or “not sure”, not the classifier itself.
Screenshots and demonstrations
There are no app screenshots here yet. PalayCheck hasn’t had its acceptance run on real phones, and I’d rather show it on a real device than in a mockup. The charts on this page are drawn from the model’s evaluation reports.
For comparison with the field results above, here is the same matrix on the held-out test set:
| BLB | Brown spot | Healthy | Blast | Sheath | Tungro | |
|---|---|---|---|---|---|---|
| BLB | 401 | 3 | 3 | 6 | 0 | 6 |
| Brown spot | 5 | 386 | 9 | 9 | 1 | 11 |
| Healthy | 1 | 1 | 200 | 2 | 0 | 4 |
| Blast | 4 | 14 | 4 | 346 | 2 | 14 |
| Sheath | 1 | 2 | 2 | 0 | 162 | 1 |
| Tungro | 4 | 6 | 7 | 4 | 0 | 273 |