On the lookout for weeds with DockWeeder

Sitting atop its four mountain-bike wheels, the Maveric makes its way across the field. With no radio control, the vehicle autonomously charts its course across the meadow, scanning every single square metre. Developed at the ZHAW School of Engineering, this unmanned ground vehicle, or UGV, has a sensor system comprising three cameras and one laser scanner. Software analyses the 3D data recorded by the UGV in real time to identify each individual plant and detect any weeds. Hot on the heels of the UGV follows a troop of robots, using precise position data to locate each weed and spray it with a jet of hot steam. That is the objective which the EU-funded DockWeeder project is expected to achieve. The ZHAW School of Engineering’s Institute of Mechatronic Systems (IMS) is one of the project partners. “In essence, this project is about recognising and interpreting images, just like the automatic face-recognition function in Facebook,” explains Dr. Dejan Seatovic of the IMS. “However, in our case the whole process is a lot more intricate, because we are not just analysing pictures, but 3D data on plants in a random environment. Indeed, a meadow presents one of the most demanding image-recognition tasks imaginable.”

Plants in permanent movement

The challenge for Dejan Seatovic and his team is to build a high-resolution sensor system which can reliably recognise individual plant types and distinguish them from others, without any human intervention at all. Only when the images have a very high degree of resolution can the data be interpreted reliably. “We are talking about resolutions of 500 dpi and above. That is nearly twice the resolution used in standard-quality printing,” says Seatovic. However, using that high degree of resolution also means generating huge quantities of data, which in turn have to be processed extremely quickly. Just one hundred metres of scanned ground data require several gigabytes of data storage. Maveric scans one square metre of terrain every second. Within that square metre, the system records one high-resolution 3D image every 20 centimetres. A further major challenge is that nature does not stay still. “Even when there is little wind, a meadow is constantly changing and the plants in it are in permanent movement,” Seatovic explains. “This means that the shutter speeds we use for taking the pictures need to be extremely fast.” In concrete terms, the images have an exposure time of 50 microseconds or 0.00005 of a second. Low-energy LEDs are used to provide the necessary illumination.

Infinite variety

The solution to these problems is based on multi-sensor systems linked to high-performance computers with machine-learning capabilities. Dean Seatovic has already successfully implemented a system which will detect rumex, a common weed. However the objective of the DockWeeder project is to develop the software to a point where it not only works for specific types of terrain and plant but is flexible enough to be used anywhere. “Every individual plant is different – which means that the problem we have to solve is more or less infinite,” says Seatovic. “Then there is the question of biodiversity. Every country and every region has plant species of its own.” Very soon, two ZHAW master’s degree students will travel to France and the Netherlands to collect additional data. The hardware they use does not necessarily have to be installed on the Maveric. It can be mounted on any agricultural vehicle.

Scope for further applications

While agriculture is not being automated as quickly as in sectors with abundant financial resources, it is still an unstoppable process. Once the software can reliably recognise individual plant types, Seatovic is convinced that it will also be used for other purposes than weeding. As he puts it, “For example, it could also be used to monitor the growth and development of plants. That information would provide a useful basis for optimising irrigation and fertilisation.”