Carrot Bot

I would like to give an evolution of the mechanical portions of carrot bot and what lynch pins allow a robust research grade platform to come together rather quickly.
1. 20 hour days. (In a weekend you can work as much as a full time job! Hell, if you include Thursday and Friday, you can work two weeks, now that is efficiency.)
2. McMaster-Carr 10AM delivery. (On Wednesday night you can get the idea to build a robot. On thursday you can order parts, get them by friday, test them out, notice something doesn't work and find a local dealer for alternatives.
3. LaserCam. (If you don't have a part just cut it out of wood, Hell even an ANSI 41 54 tooth sprocket (list price $36) can be cut in minutes for a few dollars.
4. Bike parts. They are everywhere, but some of them are broken. (no bikes were stolen in the creation of this project.)
6. Good engineers. (I don't want to tote my own horn, but...)
7. Excess stock of parts. (Everyone needs a few parts on hand every now and then.)
8. Did I mention 20 hour days. I might be delirious at this point.


Wednesday Night.



Thursday Night.



Friday Night.



Saturday Night.



Sunday Night.

She's On a Roll!

After three crazy days wreaking havoc on the biodesign lab building something that has nothing to do with biology, the rolling chassis is complete.  Now its time for Morgan to work his magic and bring her to life.

Genesis of CarrotBot

CarrotBot is a research platform for the Autonomow team.  The first task of CarrotBot is to gather data for machine vision/machine learning to test  1) Can a machine tell the difference between a weed and a plant? and 2) Can we train a machine to do this automatically?   The first version of CarrotBot will have a high-speed machine vision camera and a  high-resolution camera for visual data, and a panning LIDAR system for sub-milimeter depth measurement.  Encoders on the drive motors and RTK-GPS will measure precision position and velocity.   Assuming we validate the weed detection system, the next step for CarrotBot is a development and test platform for the weed kill system (still TBD, but looking at different methods such as clove oil, high pressure steam/water, or lasers).

Right now we're about half-way finished with the chassis, building the sensor suite and data aquisition system.  The goal is to deploy CarrotBot this Wednesday (2/9) in the fields in Avenal, CA.

None of this would be possible without the enormous help and vision (no pun intended) of Morgan Quigley, PhD candidate in the Stanford Artificial Intelligence Lab, researching robotics and machine learning.  His work can be seen here:

http://xenon.stanford.edu/~mquigley/doku.php

Trip plans for Wolrd Ag Expo in Tulare, CA

Revised plans for Wed Feb 9

- We’ll go in two vehicles (Carrortbot is large & we want to be able to separate into two groups).
- Car 1 will be van Fred will rent. Probably only two passengers and Carrotbot. (Fred - who is going with you?)
- Car 2 (my wife's CRV) will be with me, meeting at my house at 6 am. Call my cell once you get there.
- We’ll drive to Avenal first. Avenal is about 30 minutes past Coalinga in HWY 5. We should get there at about 9 am. We'll meet Malcolm Ricci from Bolthouse in the parking lot of the Burger King just East of I-5 and HWY 41, at 9 am. He'll have weedy fields for us.
- We'll all stay for 1-2 hours there. After that Car 2 will leave and go to the WAE show. Car 1 can stay as long as needed to finish collecting data with Carrotbot and then join Car 2 at WAE.
- We’ll use cell phones to cordinate. One way or anther we should meet again at the end of the day. We can fine tune times as we go.

Here is a link to the World Ag Expo site http://www.worldagexpo.com/

Jorge

Laser cutting for CarrotBot

Sat Feb 5, 2011

Interview with Ken from UC Davis

Interview with Ken Giles from UC Davis

Interviewed by Joe, Uwe and Jorge

Is automatic weeding a good idea? Yes – but it depends on the growers… organic growing is clearly a pain-point. But we need to consider labor dynamics. Hand harvesting is done by the same folks who do weeding. So if you eliminate the weeders … do you eliminate the harvesters? Ken worked in a project related to automatic harvesting and he got this feedback first hand. {Interviewer note: Automatic harvesting for “market fresh” is another interesting topic to try to address at some point. Hand harvesting is decreasing fast for processed foods – eg Bolthouse uses automatic harvesting machines, so does Morning Star (Tomatoes)}

Who is working on this? David Slaughter at UC Davis. Dutch and Denmark work … toolbar mounted more similar to conventional cultivation. Also work being done in the EU on a “swarm of vehicles” concept – lots of little machines. This was done about 8-10 years ago {Interviewer note – I know Simon Blackmore who lead this project, I’ve posted a presentation from him in Dropbox and a picture below}

Detection? First off, hand weeding is not perfect. Type I, Type II errors (desirable plants do get removed and some weeds don’t). Hand weeding is only 85 – 90% effective depending on how you count. The hyper spectral systems made by David (including x-ray sensing), in the best case, are roughly similar to the quality of manual labor. But you need very special conditions – known universe of weed species and target crop, controlled lighting.
Elimination of Weeds – you can try an organic herbicide that you can selectively place. You don’t disrupt the soil … acetic acid, heated oils, steam …
Soil disruptive techniques are also available. Underground knife, hoses, but you have to move very quickly. Hypodermic needle applicators – takes a lot of very fast actuating valves – about 5-10 ms (list price $100 per valve for the valves used at capstan – they run for about 2,000,000 cycles) – volumes are small – nano-liter range. He is thinking about use printer equipment.

Energy to eliminate weed? 1500 watts for a six inch band treatment at 2 miles per hour. Swedish folks have done a lot of work in this area. Ken will send a paper.

He offered to:

Link to paper on how much energy takes to kill a weed (Swedish paper)
Link to paper on hypodermic needle
Contact to Prof. David Slaughter

We spoke about a possible visit to UC Davis.

Interview - Niels Andrews, Coastal Tractor

Niels Andrews – Costal Tractor (Ex Eco-Dan dealer, Ex Trimble dealer).
(831) 595-3233 - cell

Interviewed by Jorge Heraud 2/4/2011


{I’ve know Niels for a while. He is well known in the Salinas Valley. He is an innovator and knows farmers and their challenges very well. He was the first dealer for Trimble’s automatic steering products, selling the first systems 10 years ago. A few years back he associated with others to represent and sell Eco-dan, a visual implement steering system to help farmers weed very close to the plants. Now he works in Coastal Tractor a Case and New Holland dealer with 3 locations}

Has been thinking about this problem for a while. Has seen several systems.
Weeding and thinning have been attempted.
Knows that T&A tried it a few years back for thinning. They spent $1M but the end result wasn’t good enough. They tried it using a green seeker looking for plant.
Blocker knife system from John Deere.
Maybe you can change the way beds are done to make the problem easier.
In Europe there is almost no labor to do this. Automation is further along in some places like Netherlands.
In the US, a few years ago when construction was booming there was almost no labor to be found for Ag.
Now there is plenty of labor, but that will be change again. This is a good time to start working on this.
It’s a great project, green solution, non-herbicide dependent.
It will require 4-5 million to do it. 1 was not enough for T&A. Sources of money: Green organizations have money too.

1. Please describe your relationship with manufacturers:
a. How do you decide what equipment you will carry?
Look at customer demand.
Is it good business. Is the manufacturer going to be there for a long term. Is the company going to support the product if there are problem.
Dealer are used to selling iron. Selling SW is harder / intangible for them.

b. What kind of margins do you get on the equipment?
It depends… 25-30%. For these margins dealer will commit to train their people and to provide first level of support.

c. Do manufacturers offer you other discounts or incentives?
Yes. It’s determined by supply and demand.
The most important thing is the product needs to be good.
Price it fairly is best. Deep discounts don’t work well as everyone finds about this, and it’s the same lowering the price but with less transparency.

d. Do you have buy-back agreements with the manufacturers?
It depends. Moving towards JIT inventory. No-one wants to own it. Dealer usually ends up owning it.
Sell a box with no brains and then sell the SW later. That way inventory costs are lower.

e. Do you have exclusivity arrangements?
What is most important is the manufacturer doesn’t sell directly. It undermines the dealers.
What has been working is revenue sharing when selling with outside your dealers territory. This tends to regulate things pretty fast.
Dealers need to support their products. This is critical

f. What type of dealer is best for this product (tractor, implement, inputs, service providers)?
Tractor dealers are best. They have deeper pockets and understand what is needed {note he is working for a tractor dealership}

2. Please describe your relationship with farmers:
a. How do you get an understanding for what equipment farmers want?

Talking with them.

b. Do you host demonstrations?
Yes.

c. Do you attend agricultural shows?
Yes. Going to WAE on Wednesday.

d. Do you provide service/maintenance for equipment?
Yes. This is key.

3. What is your advice on how a new company producing an innovative product can sell equipment to farmers?
Sell through dealers. Have product that is good and works.

Interview with FISCHER Landtechnik, Germany

Key Insights:

· Fischer would add a new product to their assortment it fits to the rest of their product line

· If the technology is new or complicated equipment is sold primarily through demonstrations. They recommend to advertise innovative solutions through magazines and trade shows (Hannover, Paris , Bologna)

· Service is done by dealer. Support and training must be provided by the manufacturer (training of 2-3 sales rep at the dealer, 2 times per year).

· 85% of all spare parts must be available within 24h. 15% of all spare parts within 72h. They keep the fast turning spare parts on stock.

· Fischer would work together with a US startup. They have no concerns regarding distance and small size of the startup.

Interview with GARTNER, Land, Forst- und Kommunaltechnik, Germany

Key Insights:

· Gartner would add a new product to their assortment if » 30% of their customers were interested. In general, they liked the idea.

· Equipment is sold primarily through demonstrations on the customer’s field

· Service is done by dealer. Support and training must be provided by the manufacturer (training of 1-3 sales rep at the dealer, 1-2 times per year depending on product upgrades).

· In Germany vegetable crops are concentrated in certain areas. We should focus primarily on dealerships in those areas.

· Gartner would work together with a US startup.

Meeting with CS experts

Jia Li – Researcher in Fei-Fei-Li with Machine Vision and Machine Learning

– Computer Vision and Machine Learning expert

Interview with Lee Redden and Jorge Heraund at Coupa Cafe

Jia agree’s that this is a very hard problem but was optimistic in that it will take about a month for a really good researcher to solve the hard parts.

She needed more images to look at but for classification if there were distinct parts of the plants that could be identified by an expert then they “should” also be identifiable by the computer.

Her algorithms in no particular order would be

- Naive bayes

- SVM

- Multiple Guesses

- Logistic regression

- Or to use a mixture of them using naïve bayes

The need for a powerful classifier in computer vision is really important.

Mike Sokolsky – Researcher with Junior

– Computer Vision and Machine Learning and Lidar expert

Interview with Lee Redden at COHO

Mike doesn’t think there is much of a business here because the labor we are replacing is pretty inexpensive and pretty good.

He had many suggestions on how the technology could look. On the algorithm side using a support vector machine algorithm as the top choice. When doing computer vision one should use a sliding window detector with plants but this algorithm might be a little slow. Another option would be to use some of the more advanced OpenCV functions such as

CascadeClassifier

Viola's Cascade of Boosted classi_ers us-

ing Haar or LBP features. Suits for de-

tecting faces, facial features and some

other objects without diverse textures.

See facedetect.cpp

His main suggestion was to drop the vision or augment the vision with a lidar system. Lidar is

- Not affected by different lighting conditions, especially the sick laser

- Creates a smaller more definiable feature set which reduces computation time. They run Junior at 10Hz. Some of the Slaughter machines run at 3Hz. And that is fast for them.

- You would not need the shroud with a LIDAR system so it is less convoluted.

- Dust on the lenses still is a problem but not as much as with a camera system.

Hokuyo makes a laser for around $1-2K that would be more prone to light irritation that the SICK but for a much lower cost.

Business Canvas Update

An update to our business canvas based on recent interviews and discoveries.

New insights:
Farmers want demonstrations.
Agricultural shows are a great marketing tool.
Dealers generally buy only 2 or 3 of any type of large equipment in a given year. Expect buy-back agreements for higher sales risk equipment.

Integrated Crop Pest Management

Integrated Crop Pest Management (sprayers for conventional farms)
Wyatt Duncan – 831-594-0435 – jduncan@redshift.com

Interview with Joe Bingold

Wyatt’s company is contracted by farmers to spray conventional fields to remove weeds and pests.

When it comes to new equipment for weed control, many farmers think up new inventions on their own and go to local fabrication facilities to have it made. “ValleyFab” is one example - http://www.northernvalleyfabricationinc.com/

Wyatt typically buys all his spraying equipment from one manufacturer – Raven Systems.

Depending on the crop, it costs anywhere from $20 to $100 an acre to spray. However, most farmers – including conventional farmers – must do additional work on their own. Most vegetable crops get weeded or thinned by hand at some point. Often, farmers will spend $500 to $600 dollars per acre in weed control by the time the growing season is done.

When it comes to new weeding equipment, the simple question in everyone’s mind is “How well does it work?” You prove that it works and everything else is easy. Distribution is not that complicated in farming. Simply get the word out and people will naturally buy to reduce their costs. If you are in the 80 to 90% range on effectively removing weeds, your system would sell very well.

Interview with Ag West Supply Equipment Distributer

Ag West Supply – Large Distributer of Farm Equipment in Oregon
Doug Hiebert – Sales and Distribution Manager – 800-842-2224

Interview with Joe Bingold

6 months in advance they need to order tractors. Can order anytime in the year.

Fall and Spring is order-writing periods for farm accessories (i.e., all large equipment other than tractors, including pull behind equipment). Manufacturer’s build most equipment on demand and have lead times as they get equipment from other vendors. It is hard to get equipment ahead of time. Generally, farmers on the west coast play second fiddle to the farmers / distributers in the mid-west who get priority from the manufacturers. (I’m guessing this is more true for the pacific northwest than it is for California.)

When a farmer asks Doug for a piece of equipment that he doesn’t carry, often there is a suitable replacement that will do the same job, which Doug does carry. Doug knows what particular equipment is suitable for their needs (primary and secondary tools) – knows what the farmer really needs.

Once a year they will have a field day where farmers and other potential customers come and watch the equipment work. Doug likes these field days, he always gets a few good orders out of the field day.

Sometimes Doug will go demo a piece of equipment at the farmer’s facility – but that happens when he knows that someone is pretty interested.

Ag West Supply has an exclusive arrangement with Case IH. Most distributers have an arrangement with one of the big manufacturers. They are allowed to sell equipment made by someone else as long as it’s not a competing project.

Doug goes to a few shows a year to find out what is new and hot in the farming community. Based on reviews from these shows, he may order a piece of equipment to try it out and demonstrate it to the farmers at the field day. Sometimes, he’ll straight-up order a few units according to what they think they can sell.

Doug won’t bring in just one piece for a customer, he needs to have a few pieces of the same type for his economies of scale to work out. He tries to turn everything in a year, sometimes material gets carried a couple years.

The manufacturer (Case IH) will sometimes help out with equipment that is not selling well.
Ag West territory is Willamette Valley – Portland to Eugene. Madras.
Vegetable farms are mostly in the Woodburn area of Oregon.

Rates for American Vegetable grower

Guys, I found price information for American Vegetable Grower magazine. This is the main one that Michael Vosti recommended. For a campaign utilizing 2/3 Page adds during 6 months it should be about $20K. I'm pretty sure we could get them to consider writing an add on our technology if we spent this kind of money.

Jorge

Pictures of an Implement

This is a tripple bar by Gonzales Machinery and Forge. It's 20 ft long like they use in Central Valley. It folds in the middle using a hydraulic cilinder and has gauge wheels for support. It uses 3 square bars - 2x 2 1/4 in and 1 x 4 in bars.

In Salinas 160" (about 13.5 ft) is more typical for implements and they don't require folding nor typically wheels.

These toolbars hey mount to the three point hitch behind the tractor.

They sell a basic 160" tripple bar for about 3K and the 20 ft for about 4K.

This is what Cliff Kirkpatrick and Doug Williams recommended when we asked.
They seem like a good place to start to me.

Jorge

NIR camera

Hi guys, I found this camera that might be interesting. Near Infrared (NIR) is used quite commonly in food processing. NIR is the energy band just below red. Red light and NIR are the primary light bands captured by plants and chlorophyl. They are used in food sorters too.

http://jp.hamamatsu.com/resources/products/sys/pdf/eng/e_c307779.pdf

Jorge

Interview with Gonzales Machine and Forge

Michael Vosti – Manager, Gonzales Machine and Forge Inc
831 675 3421 work 831 214 3840 cell
29521 Gonzales River Road, Gonzales, CA

Phone interview with Jorge Heraud

They manufacture implements and represent a few companies too. (They don’t have a website)
Manufacturing side: They build custom bars for farmers. Each farmer wants something slightly different to match what they are doing. Eg mount bars in the front belly of back of the tractor, different tools mounted to the bar, etc
Representation: Biggest one is Tatu Marchesan, a large planter and tillage company from Brazil.
According to Manta.com they sell $2.5-5M

Competitors are AgFrame in Gonzales and Sutton Agricultural Enterprises in Salinas. AgFrame represents EcoDan. He gave me the name of “Jimmy @ 831 675 2580 works for AgFrame” if we want to contact them. He said EcoDan is really not needed if you have a good driver.

He said their standard 160 inch tool bar for rear mounting in a three point hitch would be one that has 2 bars of 2 ¼ inch each. They sell for around $3K. They have also made some stronger ones that are triple ones that in addition to having the two above mentioned bars have one 4 x 4 bar with two gauge wheels. They would sell for about $4K. Staying within 2 ¼ bars means that it’s a lot easier to mount things to them as brackets start getting a lot more expensive. I asked him to send me some pictures.

If we built something like this, who would be a good dealer for it? An implement dealer that has some ability to customize, adapt and support (Eg them, AgFrame and Sutton Ag). They would probably be interested distributing it.

Tractor dealers could too, but it needs to be ready to mount. A standard machine needs to fit 95% of the users because they don’t have the ability to customize / change. Coast Tractor is a good example. http://www.coastaltractor.com/. They sell a few implements. They sell and rent.

Where do you / others find about new things?
Trade shows – World Ag Expo is the biggest. It’s not too expensive. They did a 40x40 outside booth about 6 years ago and cost them less than $1000. You spend more manning it for sure. “Everyone goes there and talks about what they saw”.

Magazines: Main ones in rank order are: Vegetable Grower, Ag Source, Farm Equipment and Tractor House.

He is planning on going to the World Ag Expo. We agreed to meet there. Should call him at his cellphone. They will be exhibiting in the Tatu Marchesan booth.

Some images from Salinas

1.Cliff Kirkpatrick, Salinas area manager for Bolthouse points to where carrots seed lines start. Lines show where he'll cultivate. Carrots are still very small. He waits until they are at 3-4 leave stage before cultivating.

2.Typical farm tractor operator has 6th grade education.

3. Picture of weeds in an unplanted and unburned area

4. Broccoli. Cultivated very close, but many small weeds remain. Weeds grow faster than plant and if unchecked will take over taking all nutrients. Field will need to be weeded by hand in a few days.

Krisada (Mick) Kritayakirana - Chris Gerdes's Control Lab

Krisada (Mick) Kritayakirana [krisadak@stanford.edu]

1^st In-Person Interview with Lee Redden

Mick is a Ph.D . researcher in Chris Gerdes’s Lab. He specializes in GPS guided control of autonomous vehicles for drifting (really cool). He just got back from a conference in Detroit on autonomous vehicles.

Images in DropBox

Mick’s main focus has been with lower level control and controllability of the car (what if you hit a mud or ice patch) and not higher level control (like is that a plant or a weed).

We talked about the accuracy of his GPS systems. He gets 2cm with RTK and 30cm with omni-star. His lab has created its own RTK system for under $4K with IMU that is similar to RTK (uses CMM for correction values). For him GPS is a tool, not what is really interesting.

Overall, Mick’s specialty was different than what we are looking into. He will introduce us to another researcher who does more image processing and CS stuff than low level control.

Reflection on asking about dollars

I wanted some input from the teaching team on this.
I have had two interviews where talking to the customer has resulted in finding a hair on fire need for a weeding system, reducing the customer cost, danger to crop from cross contamination,...(other real problems with labor) but when asked how much they would pay for a system like this, they say a number that is around 10%-20% of what they are currently paying for labor! which is too low to develop a system like this and start a company. If the number was around a 50% reduction in cost to them for labor, the company becomes reasonable. Is this common and what should we as a team think about with this kind of customer feedback?