Flowering stage detection shows ‘amazing accuracy’Research & Extension
Apple flowering stages from green tip to petal fall can now be accurately detected and mapped by machines in the first step towards the development of a primary and secondary thinning tool for tackling block variation in Australian orchards.
Researchers from the University of NSW (UNSW Sydney) and engineers from SwarmFarm Robotics who have been testing flower mapping technology on Batlow orchards this year say they can produce maps showing the current flowering stage for each individual tree from green tip through to petal fall across a whole block, and do so in close to real time with ‘absolutely amazing accuracy’.
“For the first time we can produce accurate maps and put those in front of growers and say that is exactly what stage the tree is at,” says Will McCarthy, Chief Product Office of project lead partner SwarmFarm Robotics. “We can do this, it is happening.”
Accurately detecting and mapping flower stage is the first of three intended outcomes of the Developing agri-tech solutions for the Australian Apple Industry project being delivered in partnership by SwarmFarm, UNSW Sydney and crop protection company ADAMA.
The project – a strategic investment under the Apple and Pear Fund funded by Hort Innovation using the apple and pear levy and funds from the Australian Government – also aims to produce:
• a decision support tool that will take the collated data on flower stage, tree density, height, and width and allow the grower to make a more informed decision on their spray program.
• a variable sprayer that can accurately apply thinning chemicals to individual parts of the trees within an orchard block, enabling targeted thinning of individual trees and sections of a tree within an orchard
Developing effective tools to tackle block variation and variable timing of application of chemical thinners would not only deliver big savings in removing the need for costly hand-thinning, but empower growers to manage biennial bearing, and improve consistency across the block, delivering better quality, returns and profitability.
Stage over density
Although initially planned as a three-year project to map flower density at full bloom, Will said feedback from growers had prompted a shift from flower density mapping to mapping of the actual flowering stages of each tree. Flowering stage data would have more impact on chemical thinning decisions and help manage biennial bearing, researchers were told.
“We talked to a lot of people and they told us the biggest factor in thinning decisions was timing and what they wanted to know was when to spray,” Will said. “Some said to make a difference to hand-thinning and biennial bearing, they wanted to know when the trees are at 50 per cent king bloom, so they could apply thinners at the correct stage, others wanted to know 60 per cent, or 80 per cent.
“We decided late last year to pivot from detecting white flowers to actually detecting the phenological stage and we’ve extended the project to January 2022 to give us another season to close the loop between detecting and spraying, rather than trying to do it all at once.
“We’ve proven now that we can detect the flowering stages of single trees within an orchard block from green tip all the way through to petal fall and this year we are going to improve the robustness of our algorithms as well as investigate fruitlets.”
Train and validate
The early stage maps are a simple printout that is colour-coded to show what percentage of the tree is at/or past the king bloom stage. The map below (Figure 1) shows where in the block trees are more than 60 per cent and 80 per cent at/or past king bloom stage (dark green), or at a much earlier stage (light green). The maps will be paired with density maps showing flower load in more variable blocks.
Will said this year the focus had been on training and validating the accuracy of the stage detection algorithm. “We then worked with local growers and agronomists to understand what information influenced the spray decision and the best way to present that information so it makes sense to the growers,” he said.
Over the last few seasons agronomists, UNSW researchers, SwarmFarm engineers and growers have collaborated to development robust flower stage detection algorithms by collecting terabytes of data and having industry professionals undertake field trials and cross check output of the algorithm with what is on the ground.
“We do a pass over the block to see what the system thinks is there, we then validate using the quads, then we go back to the grower and agronomist and say ‘this is what you thought was there and this is what was there’,” Will said.
“We can rapidly and aggressively build a model that is accurate. We are spending the time out in the field building the agronomy to match the technology we are developing.”
UNSW researcher Dr Mark Whitty said they had seen significant improvements in the accuracy in the detection over the three years of collecting data on flowers, and more recently individual stages.
“The work that we have been doing so far has shown absolutely amazing accuracy for what we’ve been expecting since the start of the season,” Mark said. “The ability to detect the growth stage during flowering is a world first.
“We’ve also been able to speed up the process from something that is able to run offline for many hours to something that is able to run very close to real time to give the farmers an on the spot idea of what is happening in the blocks.
“They can then take that information to go out with the variable time of application sprayer to actually thin chemically in order to get their consistent performance of the trees.”
Will said growers were critical to the validation process, bringing experience and gutfeel.
“We could build a model that says 80pc of king bloom and it could be technically accurate, but you still need the algorithms to align with the grower’s expectations,” he said. “That’s why we use people that look at the tree every day to do the field work and advise us on training and the output. Growers know their own orchard and really want a tool to give them the confidence their decisions are correct, or assist them in making the right one.
“The most exciting part about this year was when we first output stage maps and sat down with growers to say this is exactly what is on the trees, and they looked at it and said ‘yes, that is what we are seeing’.
“That’s the validation we want, it is really the farmer feeling.”
The trial and development to date has been done in Pink Lady and Gala blocks.
Will concedes the more data you can get on individual varieties the better the accuracy, but said the early focus was on proving it could be done.
“The feedback we had was if you can do Pinks and Gala well, they are the two main priorities,” he said. “As this technology is commercialised and it gets to industry, we can see how adding other varieties looks.”
Closing the loop
Now the team is confident they can accurately detect and map flowering stages, Will said work would step up on closing the loop between flower detection and spraying starting with working with growers to use the maps to make decisions on the ground.
SwarmFarm and ADAMA will be looking to pair up with industry-linked companies to integrate the decision support tools.
After flowering, spray trials will resume using a tower spray rig built for the project by spraying system specialists Silvan Australia capable of independently controlling applications between the top and bottom of the canopy.
Will said with the emphasis of the project shifting to timing of application, the spray trials would be on accuracy of delivery rather than variable rates.
“At this stage we are not changing the rate,” he said. “We are changing when it is delivered. It will be a Variable Timing of Application (VTA) thinning tool.
“Variable timing is about targeting each individual tree and applying thinning products at the perfect time for each individual tree.”
Commercialising the new variable timing technology so growers can access it is a priority and it is expected that the three components would be commercialised with partners with existing expertise and growers would be able to opt for the tier of variable spray automation and mix that suited their individual preference.
“We want to get this to growers,” Will said. “We signed a memorandum of understanding between ourselves and Green Atlas last year to co-operatively advance precision thinning, so this system would add on well and advance their existing commercial product. The sprayer has been provided by Silvan Australian and we have worked together to achieve individual tree-targeted spraying.”
“Growers who don’t want to spend money and are happy to manually turn the sprayer on and off might go for an iPad app, the second tier might be a variable rate sprayer sitting behind the tractor and the third tier of delivery would be a full robotic solution, where the robotic would go and collect and map the data and operate the sprayers at different levels.”
Automation and variable rate spray close to delivery (October 2019)
New project to deliver agri-tech solutions (August 2018)