How Dogtooth Technologies' intelligent robots are mastering strawberry-picking
Machine learning and computer vision combine to help solve labour shortage on farms
Intelligent robots could have an even greater impact on society than the internet. That’s the view of Dr Duncan Robertson, the chief executive officer and co-founder of Dogtooth Technologies.
The Melbourn-based start-up, a nominee in the Cambridge Independent Science and Technology Awards, is developing a new fleet of fruit-picking robots that will navigate autonomously down polytunnels, find ripe strawberries, carefully pick them, perform a visual inspection and place those suitable for sale in a punnet.
“Robots, of course, have been around for decades,” says Dr Robertson. “When we talk about intelligent robots, we’re talking about a new breed of robot capable, in some sense, of making its own decisions about what to do and when to do it. The robots that build our cars follow the same routine day in, day out. By contrast, our robots make a continuous stream of observations about the world around them and decide what to do next in order to maximise performance.”
It is only thanks to advances in machine learning and computer vision that a strawberry-picking robot is feasible.
Dr Robertson, who has a PhD in these subjects from Cambridge, says: “Our robots need to do a very similar job to human pickers. They are designed to work on the same farms and to be compatible with the same infrastructure. They are picking fruit grown using the same tabletop-growing systems.
“They are the right width to fit into the aisles of the same polytunnels.
“The robot navigates autonomously and searches for fruit that is ready for picking.
“It then determines how to pick that fruit and uses a special robot arm end effector to both cut the fruit from the plant and grip it securely.”
Dogtooth’s patent-pending approach snips the stalk just above the leafy calyx – which, incidentally, is edible and contains antioxidants – at the top of the strawberry.
Some rival robotics manufacturers are trying other methods – Belgian company Octinion’s robot grips the fruit between cushioned plastic paws and twists it to snap it off without a stalk, for example.
But UK supermarkets are accustomed to selling strawberries with a little stalk left, as this is what customers have come to expect.
Dr Robertson is confident that the robot will match human pickers in selecting ripe fruits.
“In strawberries, colour is an incredibly good indicator of ripeness, and we can evaluate the colour of the fruit. We might not always get it right when the part of the berry that we can’t see is unripe but we get it right a very large proportion of the time,” he says.
And there are areas where robotic picking proves more efficient.
“One of the exciting aspects of our technology is we perform an inspection step during the course of picking,” says Dr Robertson.
“Having picked the fruit, we position it in an inspection chamber and image it from all viewpoints all around the fruit, in order to determine its shape grade, to measure its mass and to detect various kinds of defects that might make it unsuitable for retail, like bruising or mildew. Having done this, we then place the fruit in the appropriate punnet or in a waste container.
“With human pickers, the punnets are typically post-processed in a packhouse and among other things they redistribute from punnet to punnet to achieve minimum weights. We don’t need to do that.
“We deliver punnets that give or take a cellophane wrapper are ready to be shipped to the supermarket and that’s one of the bigger wins of robotic picking.”
Others benefits include the potential to pick fruit at night, when cooler temperatures could aid picked fruit’s shelf life, and the ability to collect data about crop yield.
Another advantage is hygiene.
“An interesting challenge for human pickers is to avoid cross-contamination. In handling fruit it’s very easy to transfer disease pathogens from one berry to the next – pathogens like fungal spores in the case of mildew,” says Dr Robertson.
“But the robot picker picks fruit by the stalk so never handles the body of the fruit and no part of the body of the fruit ever touches any surface of our robot.”
Should any fruit prove unsuitable, it is deposited in a waste container and taken off the farm.
“We don’t leave rotten fruit because that’s bad for the health of the plant,” explains Dr Robertson.
Of course building a fruit-picking robot in a controlled environment is hard enough, but deploying a fleet on a farm is quite another, so Dogtooth has been testing its four version two robots for the last year with growers in UK and Australia – giving them a year round picking season.
“We are standing on the shoulders of giants here. This is testament to enormous progress in academia and beyond in the field of computer vision, particularly in areas like semantic image understanding, which means teaching computers to understand the world using images, and policy learning – learning robot-control policies for efficient operation using training data derived from real world experience or simulation,” says Dr Robertson.
“One of the ways Dogtooth can claim to be a very pioneering tech start-up is in taking a holistic view of the challenges that must be overcome before intelligent robots can be brought to bear on real-world problems.
“Making robots that work well enough is very difficult. Robots are very complex systems and we can only build them by combining knowledge of a wide variety of engineering disciplines.
“We have a really first-rate team at Dogtooth. We combine expertise in software engineering, electronics engineering, mechanical engineering and so on. What we do is made possible because of enormous progress in machine learning research. I like to think that we are ahead of the curve in that domain.
“But it’s often the more basic, practical problems that matter more than the clever technology. It’s the question of how we deploy a fleet or robots in the pouring rain on a muddy hillside in Kent. Those challenges are just as significant, if not more so, than the deep technology challenges.
“Like all good engineering teams, we build prototypes, learn about them and then make a better version.
“We’re focused on strawberries at the moment but will later turn our attention to raspberries and blackberries. And I often receive emails from vineyards interested in what we can do to help them.”
Dr Robertson says the company, which has raised around £6million in funding, has gone “a very long way” to addressing these practical deployment challenges, thanks to a “brilliant engineering team”.
He adds: “Strawberries are grown under plastic in polytunnels so we don’t need to build completely waterproof robots but yes, they do need to survive the rain and pressure washing, as they get muddy, and deal with difficult terrain. We’re very pleased with the performance of our robots in these kinds of environment.”
Dogtooth is about to manufacture 24 of its version 3.0 robots, which means that some of the strawberries you buy next year may never have been touched by a human hand.
The interaction between man and machine remains a key consideration for Dogtooth, however.
“Apart from efficacy, another important challenge is the health and safety, or regulatory, one. We are building robots capable of moving under their own control in the vicinity of people. It’s important that we think very carefully about hazards that this can create.
“Conventional industrial robots are typically bolted to the ground and typically works in a cage so is made inherently safe. We can’t do that.
“We’ve been working hard on building safety into our robots so we can sell CE-certified machines. It’s something a lot of technologists have been prone to forgetting in recent years.
“But none of these things matter unless we can address a third challenge – and that’s to build robots cost-effectively.
“If you have a million pounds you can buy a robot that can peel an orange. The problem is nobody needs a robot that can peel an orange – and certainly not for a million pounds.
“By contrast, we are building robots that need to do the work of people and they need to do that work on a comparably or more cost-effective basis. It’s not something we can retrofit. We’ve had to design into our robots from the outset the possibility of scalable, mass production.
“We see ourselves as a bit like the Henry Ford of intelligent robotics. We want to build robots in large numbers and very cheaply.”
It is likely that Dogtooth will initially sell picking services, using the market rate for employing human labour. But buying a robot may one day be comparable to buying any other substantial piece of farm machinery.
Dr Robertson can’t yet disclose how much it would cost to buy one of Dogtooth’s fruit-picking robots, due to commercial sensitivity. There is a robotic arms race on, and the company intends to compete globally in a market for picking labour worth about £1billion.
“You don’t have to look far to find predictions that intelligent robotics will be worth hundreds of billions and ultimately trillions and I support that view,” says Dr Robertson.
“I think that intelligent robotics will be more transformative in its impact on our society than the internet was.
“However, that’s not going to happen unless companies like Dogtooth take a holistic view of some of these key barriers to exploiting this market.”
Dr Robertson saw the opportunity to do so back four years ago along with co-founder Ed Herbert.
“Ed and I were becoming rather frustrated that innovation in those domains seemed to be largely constrained to mobile phone apps.
Seeking something more tangible, robotics provided the perfect application.
“Soft fruit picking ticked the box that every tech start-up is looking for: It’s neither so easy that everyone was doing it, or so difficult that we didn’t believe it to be amenable to the expertise and IP that we could bring to the table.
“Since then we’ve built great relationships with a number of customers who have helped us build a product that we expect to be very relevant to the needs of growers around the world.”
And the name? It reflects that these robots can hold fruit in their jaws – and have the potential to become “faithful servants of humanity”.
Robots ‘can help solve farm labour shortage’
Amid much discussion about how the world of work will be disrupted by robotics and automation, Dr Duncan Robertson stresses that the agricultural industry needs solutions to its labour shortage.
“Our goal is not to displace people,” he says. “We don’t want to take jobs away from people. We are in this business because growers in this country and elsewhere can’t recruit enough people. We are focused on filling the labour shortage, and equipping the people we do have with the power tools to do their jobs more effectively.
As skills – and expectations – of workers rise, recruiting to traditional manual roles like fruit-picking is proving challenging. And in this country at least, the situation could get worse quickly.
“Brexit exacerbates the problem in Britain, but we’re a global business and we take a broader view,” notes Dr Robertson.
“I don’t think many of us will have missed the headlines about the labour shortfall in the UK agricultural industry.
“We hear the same thing from growers around the world. We work with pickers in California who tell us they can’t recruit enough pickers. It’s the same in Europe and even in China.”
From Microsoft to online fashion
Prior to focusing on fruit-picking robotics, Dr Duncan Robertson’s expertise were being used by the likes of Microsoft Research and local online fashion start-up Metail.
Having completed his PhD in machine learning and computer vision in 2003 at the University of Cambridge’s Department of Engineering, Dr Robertson co-founded Redimension with his supervisor Professor Roberto Cipolla and Oxford University alumni Dr Geoff Cross in 2004.
The company offers consultancy services for those looking to exploit computer vision and has worked with clients including Microsoft Research and Samsung as well as number of technology start-ups. Dr Robertson was also a co-founder of local online fashion startup Metail, who sell technology to allow consumers to use accurate 3D body models to find outfits that will suit.
Read more
CMR Surgical raises record $100million - and hopes to have its robots in hospitals by end of 2018
Meet the robot that helps you learn from afar
Charles the robot head and the face of human emotion
