Lose It app promises to log your meal just by taking its photo

Keeping track of what you eat has proved to be pretty effective at aiding weight loss. Studies show that food diaries not only help people manage their daily caloric intake, but also be generally more aware of what they're putting in their bodies. Unfortunately, logging every meal can be tedious and time-consuming. But what if you could do it just by taking a photo of your food? That's exactly what Lose It, a food-tracking app, is trying to do with a brand new feature called Snap It. Using a combination of machine learning and its own food database, Lose It aspires to figure out what you're eating based on your photo alone.

As a warning, the feature is still in beta, so it's not perfect. For one thing, it's not fully automated; you can't just take a photo and get information on exactly what's in the food. But for what it is, Snap It comes close. After you take a photo of what you're eating, the app analyzes the image and spits out a list of suggestions of what it thinks it is. Pick the option that fits best, and then you'll be brought to a screen where you can add more details, such as whether that piece of fried chicken was a thigh or a breast and how much of it you ate. If there are multiple foods on the plate or if the app just didn't guess the food correctly, you can also just enter it manually via the Add Food button at the bottom of the photo.

I tried out the app for a week, and while it didn't always recognize my food, it fared well enough for me to come away impressed. I found that the list of suggestions based on the photos almost always included at least one correct answer. When I took a photo of a bucket of fried chicken, for example, the very first suggestion was "fried chicken," followed by "chicken thigh" and "pork chop." OK, that last one wasn't right, but it was still a somewhat understandable guess. The same thing happened when I took a photo of strawberries -- Lose It recognized my snack on the first try.

Things got trickier when the photo included multiple foods. In one instance, I attempted to log a plate of chicken, collard greens and mashed potatoes. The app could spot the chicken and the mashed potatoes, but not the greens. But in a photo of fried rice, spinach and chicken, the app was able to recognize all three instantly. In yet another picture of a culotte steak drenched in a cheese-based sauce, the app was pretty stumped as to what the sauce was, but did recognize that there was a steak. And it missed the arugula and tomatoes that were underneath the steak altogether, because, of course, they weren't visible on camera.

It's issues like this that make food logging via photography such an inexact science. A photo of a bowl of curry won't be enough for you to figure out exactly what kind of vegetables and spices are in it, and just looking at a salad dressing won't help you tell you if it has sugar. "That's why we're doing this semiautomated to start," says CEO Charles Teague. "The idea that you could look at a picture and instantly know what it is, it just wasn't going to work all the time."

With the assumption that it was never going to be 100 percent accurate, the goal of the Snap It feature, at least for now, is simply to make it easier to log your meals. And I have to say, I found it was. Prior to using the app, I wasn't a fan of keeping a food diary precisely because it seemed like a hassle. But using the camera to snap my food and having a little automation made it simple enough that I now find myself logging my meals all the time.

That, Teague says, was the original point of Lose It. The company set up shop about eight years ago with around 50,000 foods in its database. Now, it has millions of entries. Recently, Lose It added the ability to add foods by scanning a bar code. It even has location services to see if you're within walking distance of a restaurant it recognizes -- usually a chain -- and when you go to make an entry in the app, it'll instantly suggest the kinds of foods you can get at that restaurant. The next step is machine learning, and though the Snap It feature is still rough around the edges, it has promise.

1 / 6

"Our strategy is great because when we start getting all these photos of food, it becomes a dataset we can use," Teague says. "We'll have photo data, food data and eventually, location data. There'll be a lot more context around the user that can make the app a lot smarter." So even if a photo might not indicate that a dish is a curry, for example, the fact that it was taken in the vicinity of an Indian restaurant might teach the app to at least suggest it as a possibility. "It could be the combination of the photo and the location that could reveal very specifically what it is."

The food identification might be semiautomated now, but Teague seems optimistic about what the technology will eventually be able to do. "Our expectations are that this will generate huge amounts of data that we can use to continue training and improving the machine learning," he says. "That's going to drive more accuracy in what we recognize, and the ability to recognize even more things.

"Each photo that you log will become a piece of data that we use to train the next generation of the app," he says. "We might even be able to estimate serving sizes."