The Future of Online Ratings and Reviews
The cute girl from last Friday’s Christmas party agreed to go on a date with you! Nice work, my friend! Time to find a good spot to grab dinner. If you’re like the rest of my net-savvy generation, you pull up Yelp and start looking for places. The list of restaurants come up and as you’re scrolling down, you can’t really tell the difference between their ratings. This sushi restaurant has 3.7 stars, but this Italian restaurant has 3.8 stars and the Mexican place down the street has 3.6 stars. Is a 3.7 star sushi restaurant really that much better than a 3.6 star Mexican place? Okay, time to check out what people had to say in their reviews. One person says they got food poisoning from this place, but another says they proposed to their wife there and another said it’s just okay, but not better than the place around the corner. Who do you trust?
There is a fundamental problem with the online rating systems we use to give and find feedback on businesses and products. I’m going to dive into these problems below and try to give a little insight into how I believe we can solve them. Then I’ll try to provide a glimpse of the vision I have for the future of rating systems and how we use them to discover the world around us.
The Problem with 5 Stars and Written Reviews
The most common rating system we use online is the 5-star scale. This scale was adapted from the familiar ratings we found in newspapers growing up. Newspaper critics helped people discover the best new restaurants, movies, plays and events by rating them and justifying that rating with a written review. When the internet began implementing feedback mechanisms, the 5-star and written review system made the most sense because it was familiar. Websites could average out the ratings and feature written reviews that users found the most helpful. It worked pretty well in the beginning, but then as more and more people began rating and reviewing things, the utility of that feedback became murky at best.
The problem that began to emerge with 5-star ratings is that as more people rate things, the less differentiated those things become from the rest of the market. The clearest example of this problem emerges when searching Yelp in cities where most businesses have over 1,000 ratings. When the search results populate, you’re going to be stuck looking at a range of ratings from 3.5 to 4.2 stars generally. The 5-star scale was meant for critics to provide a singular opinion. It was not designed to be averaged across thousands of opinions. It also wasn’t designed for amateur reviewers because we tend to rate things at the extreme ends of the scale.
YouTube realized this problem after they analyzed the distribution of user-ratings when they had a 5-star scale below every video. The graph above shows that users were much more likely to give videos 5 stars or even 1 star than to bother with the nuance of giving a 2, 3, or 4 star rating. Since sharing this graph in a blog post, YouTube moved to a simple thumbs up or down. Kudos to them for realizing they made a mistake and fixing it!
There is also the problem that everyone has a different interpretation of the 5-star scale itself. Some people might hand out 5-star ratings like candy on Halloween and some might hold onto one full-size candy bar for the best costume of the night, yet both of those people are counted equally when averaged. The subjectivity 5-star scales inject into the rating process even further decreases the usefulness of the average ratings.
If we can’t tell the difference between businesses based on their ratings, the next source we generally turn to is reading the reviews other people have written. Written reviews suffer from the same problems as the 5-star scale though – they are highly subjective. When you ask your friends how much they like a business or product, you know who’s opinion to trust (and probably don’t even ask the people you don’t trust). There is no way to tell who’s opinion to trust and who’s to discard online. Is the restaurant really that dirty or is the person who wrote that review an OCD germaphobe? Did this customer really receive poor service or was it written by the owner of a competitor down the street to boost their own business? Again, who do you trust?
The Solution is Simplicity
How do we solve the rating system problems described above? KISS! Keep It Simple, Silly! Okay, not THAT simple. The en vogue thing in social media right now is to use an over-simplified system where users just “like” or “+1″ things. It seems like it should work – the more people who “like” something, the better it is, right? Wrong. When you throw out the ability for people to express dislike or negative experiences, you create a system that doesn’t reflect reality. What does a “like” or “+1″ really mean if people don’t have the option of “disliking” or “-1″ something?
So maybe instead of KISS, it should be KIPS. Keep It Pretty Simple. Simplify the rating system into a straight-forward measure of like and dislike. This rating system approach has gained popularity with the emergence of websites like RottenTomatoes and Reddit. RottenTomatoes is an interesting case because they have developed a system to convert ratings into a binary positive/negative rating. They knew that if they just averaged all of the 5-star ratings for movies across the country, their website would be full of movies with 3.5-4.2 stars. Instead, the movies currently in theaters have scores ranging from How to Survive a Plague’s 100% fresh rating to Playing for Keeps’ rotten 3%. While RottenTomatoes aggregates movie reviews from vetted sources, Reddit provides a model where the rating process is given over to the community. Every user has the ability to rate any link submitted to Reddit by giving it an upvote or downvote and links are ranked by their net upvotes (upvotes minus downvotes). Reddit is quickly becoming one of the most influential sites online and it’s popularity is highly attributable to the beautifully simple rating system at the core of its functionality.
Another part of the solution for improving our rating systems online is the integration of social media’s secret sauce, photos. Photos have been the fuel that drives websites and apps like Facebook, Reddit, Instagram and FoodSpotting. At the end of the day, we are very visual creatures. It makes sense then that any solution for improving rating systems should pay special attention to photos. Anyone who has used FoodSpotting understands the maxim “a picture is worth a thousand words” is especially applicable when choosing where to eat. Yelp understands this and has begun to build photos as a central part of their experience, but they are still chained to their flawed 5-star ratings and written reviews.
The Future is Rating Pad Thai
If rating systems are already slowly moving towards a pretty simple binary rating scale augmented with user-submitted photos, where is the next point of disruption and opportunity for innovation? I believe the future of rating systems hinges on our ability to add layers to our experience. Right now, we go to a restaurant, come home, and write a couple paragraphs describing what we ate, the service and the overall ambiance. The rating we give is meant to describe your overall experience with that restaurant, which, similar to “likes” and “+1″, is a little too simple. Maybe you went to a restaurant, loved the ambiance and food, but your server was new and in over their head? Life is complicated and our experiences are influenced by a number of different factors. I believe the rating systems of the future will let you rate each of those complicating factors separately, so you can give the food and ambiance of the restaurant a positive rating, but give the service a negative rating.
Adding these layers respects the complexity of our experiences and provides more useful insight to people who use the ratings to make decisions. If someone has a hankering for pad thai, they can search for the highest rated pad thai dish near them instead of just hoping the highest rated thai restaurant has the best pad thai. It lets people search for the best masseuse instead of the best spa and the best beer instead of the best bar. Adding layers opens up a whole new world of social discovery when combined with a simplified rating system and photos.
SXSW Schedule
If you want to meet up at any point in the weekend, here’s a schedule of events where you can probably find me:
Friday
2:00pm – Making a Grant Entrance: How to Launch a Product
3:30pm – Show Me the Money: Where to Get Funding and What Are They Interested In?
5:00pm – Altimeter Group Cocktails at The Brown Bar
8:00pm – Techset Party at The Parish
Midnight – Party-hopping
Saturday
9:30am – Help Wanted: Hunting High & Low for Digital Talent
11:00am – (Not Just) Shit Startups Say: Anatomy of a Rockstar Product Cycle
2:00pm – SXSW Co-Founder Speed Dating
3:30pm – Finding Co-Founders: the Who, Why, Where & How
5:00pm – PARTIES! Find me on foursquare
Sunday
11:00am – Startup Breakthrough: Speed Coaching for Founders
12:30am – Adding Value As a Non-Technical No Talent Assclown
2:00pm – Allhat 4: Mootown’s Greatest Hits
3:30pm – Investor Spotting 2012
5:00pm – Big Data: Privacy Threat or Business Model?
7:00pm – MORE PARTIES! Find me on foursquare
Monday
11:00am – Wonder Woman (documentary)
2:00pm – Expanding Our Intelligence Without Limit (Ray Kurzweil Keynote!!!)
3:30pm – The End of Business As Usual (Brian Solis Keynote, with special guest Billy Corgan, lead singer of the Smashing Pumpkins!!!)
6:30pm – Pitch Some Good
8:00pm – PARTIES!…. you get the pattern here.
This schedule is highly subject to change, so shoot me a tweet at @Zak_Kirchner or @digthevibe if you want to meet up. If you know of any parties, events or sessions that are can’t miss, let me know! I’m keeping my expectations low for the weekend and trying to go with the flow.
Good Vibes.
Digging the Vibe at SXSW
Greetings from Austin! I’m here for the SXSW conference on behalf of the startup I’m co-founding, digthevibe. I’m incredibly lucky that my employer, Altimeter Group, has been so supportive throughout the pre-launch/funding phase of our development, but I’m also really excited to take a week of vacation and focus 100% of my energy on digthevibe.
While here, I’m going to be attending a number of sessions and will be live-blogging them on digthevibe’s blog. Sessions only convey a small portion of the value that SXSW has to offer, so I’ll also be attending a number of events, parties, tweetups, and networking like a mad man.
If you want to connect this week, there are a number of ways to get my attention. You can email me at zak (at) digthevibe.com, tweet me at @Zak_Kirchner, find me on foursquare or write on my Facebook wall.
Please also take a second to “like” digthevibe on Facebook. I’ll write a full post explaining what I’m doing at digthevibe and how we’re going to change the world, but for now you’ll just have to track me down and hear the pitch in person. One of my top goals for the weekend is to refine our story and pitch for potential investors.
Speaking of, we are looking for funding… a lot of funding. digthevibe is a very ambitious idea with a potentially explosive business model and we’re going to need a lot of money to pull it off. If you know any investors who I should talk to, I would love to meet them.
Breaking Through the Noise with Content Marketing
I am VERY excited to announce that Altimeter Group‘s newest report, Content: The New Marketing Equation, has been released to the public. It’s Rebecca Lieb’s first report with Altimeter Group and the first report that Jaimy Syzmanski and I supported as researchers. We conducted 56 interviews with content marketing professionals and used those insights to develop a maturity model and audit for organizations to benchmark their performance.
Five stages of content marketing maturity:
- Stand - Curiosity and consideration – Not yet practicing content marketing
- Stretch - Advocacy & experimentation – Begin to build strategy and support to publish content
- Walk - Strategy & Processes - Solid organizational and strategic foundation, teams formed, metrics introduced
- Jog - “Culture of content” – Sustainable, meaningful, scalable content initiatives, broad training organizationally, top-down and bottom-up
- Run - Inspired and inspirational (and largely aspirational) – The company actually monetizes its content, which has a separate P&L
If you’re curious where your organization falls in this maturity model, read the report embedded below and answer the maturity model audit on page 13. All of Altimeter’s research is released under our Open Research policy, so feel free to download the report and share it however you see fit.
What I Personally Learned From This Research Process:
- Content breaks through the noise. Marketers face a huge problem right now because consumers have access to more information online than they’ve ever had before. Breaking through the noise online is incredibly difficult, but I think content marketers are on the right path. I’ve never heard marketers focus so much on solving problems for their audience, which was a breath of fresh air. The content produced for a brand shouldn’t promote a product, which is counterintuitive for marketers. Instead, it should focus on solving customer pain-points and telling the story around the product, not about it.
- Story-tellers are a hot commodity. Nearly everyone we interviewed emphasized how important it was to tell their brand’s story. Every brand, product and employee has a story to tell, but not every organization has hired people with the skill set to actually tell those stories. Brands are looking to hire journalists because their skill set aligns very strongly with the talents they need to produce a high volume of quality content.
- That film degree might come in handy. As brands move to more visual content, artists and filmmakers will see their demand increase as well. Journalists and editors are the primary hiring target for organizations right now because they need people to produce a steady stream of blog posts, but online video is a high priority for content producers and they’re just starting to figure out how to do it quickly and professionally.
- You get what you pay for. Like social media (and social media is often a large chunk of content marketing), people still think about content marketing as being free. They don’t see the costs associated with writing blog posts, managing twitter conversations, replying to Facebook comments. The channels might be free, but managing and producing content for them is not. You can tell a lot about an organization’s priorities by looking at their budget. We’re seeing more budget dedicated to content marketing, but it’s still too small a slice for the value it produces compared to traditional media spending.
Read and download the full report here:
The Future of Geo-Social Apps – Live Blog from Social-Loco
This is my first attempt at live-blogging a panel at a conference. I’ve copied the notes I took below with very few edits to give you an authentic feel for what it was like to be in the audience at Social-Loco. I apologize if they’re a little discombobulated, but that kind of comes with the territory of 6 people talking at the same time on stage.
The Future of Geo-Social Apps
Moderator: Di-Ann Eisnor, VP, Communtiy Geographer, Waze
Panel:
- Ian Heidt, Product Lead, Neer
- Sam Altman, CEO, Loopt
- Alexa Andrzejewski, CEO, Foodspotting
- Robert Scoble, Managing Director, Rackspace
- Andreas Winckler S. Advanced Technology Engineer, BMW Group
It all starts with context.
- It’s before you go out. Can it help you find what you want? (has to go beyond search though)
- Then once you’re at a place, what do you do? Can it help you figure out what to do when you get there?
- Third is sharing your experience with other people.
Notes:
- Location-Based Apps (LBA) help you when you’re searching for answers in a foreign environment. Those are the situations where LBA can be magical.
- There is a lot more potential for LBA in cars.
- All of the censors being integrated into cars can add a lot to the driving experience.
- The censors in your gas tank can power an app that intelligently suggests refueling options based on proximity and price.
- The camera is one of the biggest censors for delivering context in any device, including cars now.
- How do we take all these inputs and make them “human-understandable”?
- We’ll start linking emotional state to locations. How are you feeling in a place? Do you dig the vibe?
- The camera drives augmented reality.
- Color is an app that leverages these techniques. It’s aware of the devices around it. It maps the audio of a room.
- There are multiple ways to get these answers around mobile social. Battery is the killer. What’s the most efficient?
- You can use accelerometers to count steps, audio to recognize ambient noise, etc. What’s the most useful and efficient use of your device’s limited capabilities?
- Mobile devices aren’t just used on the go. They’re used in our homes. How can we add value to the home experience?
- What about location-tracking? Privacy?
- Companies have been tracking our cell phones forever. They tracked OJ in 1992 by his cell phone.
- Personalized control. Balancing user-customization with ease of use can be difficult because privacy fluctuates greatly in different contexts.
- Foursquare is integrating everywhere, which is why it isn’t over-hyped.
- We’re wasting a lot of energy because we haven’t optimized our traffic experience, leveraging data about traffic jams, energy
- You can compete with your friends about who can save energy.
- Only 21% of people are motivated by gaming mechanics – limited potential there.
- We’re refining our understanding of game-mechanics and adapting to what’s useful to people.
- Game-mechanics are a great way to jumpstart a service, but it needs to move the product forward to realize its potential.
Moving to San Francisco – The Online Tools That Helped Make My Relocation Easier
Hire movers. They're worth every penny.
In the middle of February I received an offer to work for Altimeter Group in San Francisco, CA. This was an excellent opportunity to take the next step in my career and work with a very talented, intelligent group of social media experts. The only thing that kept my excitement in check was the prospect of relocating to a city I had never visited before. The first time I ever set foot in San Francisco was actually for the last round of interviews, so moving somewhere so unfamiliar was daunting to say the least.
I had already made a major relocation once before when I moved cross-country from Chicago to Los Angeles three years ago, but that move wasn’t as intimidating because I had a very solid base of friends who moved to LA after graduation. Three of my four best friends lived there along with a large number of friends I had from the theater department at Northwestern University who were trying to succeed in the entertainment industry. It was stressful, but having a network of friends there made the transition much easier.
I knew three people in San Francisco before I moved here and one of them was studying in Japan for a semester. That coupled with the fact that I had never set foot in the city meant I needed to get creative and use every single tool at my disposal to make it work. The following is a list of the most important tools, websites and apps that helped me make this transition as smooth as possible.
Leverage Your Networks
This might seem obvious, but it was an important first step for me so I had to include it. Hopefully you know someone in the city you’re relocating to, and if not, then hopefully the rest of the tools I list will help you. The biggest question mark for me when moving to San Francisco was where to live. Every neighborhood has it’s own character, traditions and style, but I only had two weeks to find an apartment and move in, so the first thing I did was reach out to the three people I knew there to ask them which neighborhoods I should look at in my apartment search. This helped immensely. Even if you don’t know someone in the city you’re moving to, you probably know someone who grew up there or lived there at one point. One tool you can use to help this is searching your friends list on Facebook by hometown and current town.
All I can say is thank God for Craigslist. I can’t imagine apartment hunting without it. I was also lucky that the San Francisco Craigslist pages allowed you to narrow your search for apartments by neighborhoods. This feature is available in New York City, but not Chicago or LA. You can always work around it by searching for the neighborhood’s name, but it can be tough when neighborhoods have nicknames or if the posts only mention a street.
UPDATE (2/17/12): @JulesFaas made a great suggestion in the comments to use PadMapper. Couldn’t agree more with her. One of the more frustrating things when comparing Craigslist apartments is that you can’t compare locations on a map. For those like me who are more spatially and visually inclined, this website really helps augment the Craigslist experience.
One of the best new websites I discovered in this process was WalkScore, which rates neighborhoods on a hundred point scale based on how walkable they are. Scores under 50 mean you’ll be dependent on a car and anything above a 90 is a “Walker’s Paradise” where your daily errands don’t require a car. It was really important to me that I lived somewhere very walkable, so this was incredibly helpful. It also provides you with lists of restaurants, bars, coffee shops, grocery stores, parks, and other lists of places nearby that can give you a better idea of what’s in the neighborhood. My current apartment’s score is a 97 and I’m thinking about getting rid of my car now. My old apartment in Venice was an 78, which is “Very Walkable”, but the difference between the neighborhoods has definitely made my move much easier. I LOVE how convenient my new neighborhood is and the scores pretty accurately reflects that convenience. Even if you aren’t apartment hunting, it’s interesting to see how your neighborhood scores.
Craigslist and WalkScore gave me a bird’s eye view of possible apartments and what their neighborhoods has to offer, but Google Street View rounded out the experience by giving my the ability to virtually walk down the street. Almost all apartments on Craigslist include a link to Google Maps and from there you can zoom in far enough and actually move around the neighborhood as if you were just out for a walk. It’s a great tool for getting a feel for what the neighborhood looks like without ever stepping foot in the city. This was especially helpful in my search considering how little experience I had with San Francisco before moving here.
The second weekend after I got my offer I drove up to San Francisco to check out apartments. I had been looking at Craigslist nearly every day for potential apartments, but decided the best way to save my favorite ones for follow up was syncing them with my favorite note-taking application, Evernote. Using the Chrome plug-in, I clipped full page craigslist ads to my account, which synced with the app I had installed on my iPhone. That way I had access to the pictures, links, phone numbers and addresses in each post on my phone without having to re-search Craigslist. It’s true you could create a folder of bookmarks and then sync it to your phone’s web browser, but Evernote allows you to write your own notes above the webpage that’s been clipped. So after I’d check out an apartment I’d give it a rating and type in any important things like “Windows look out onto an ugly brick wall. Feels like a prison.” or “Landlord is really friendly and street parking nearby is pretty easy”. If you aren’t using Evernote yet, I’d strongly recommend checking it out. It’s simply the best note-taking application I’ve used yet.
Apartment Application + Credit Report
One of the most annoying parts of any apartment search is filling out multiple applications that ask for the same details. Knowing that I was going to check out at least 10 apartments and that I might want to apply for up to five or more of them, I decided I’d create my own application. I google searched “apartment application” and found this document. Applications can vary between management companies though, so I just used it as a guideline and created word documents for each section. Looking at a couple applications will let you know what to include in the document, but mine included previous addresses (including the dates I lived there, management company contact information and monthly rent), my employment history, professional contacts and personal references. I also included a credit report so I could avoid some of the application fees, but most management companies will require you to submit to a background/credit check anyway. Only two management companies accepted my credit report, but it only cost me $15 and ended up saving me about $40. Put all these documents into an envelope that you can hand to the landlord when he’s showing you the apartment and you come off as very prepared and responsible. Plus, it’s great to just have all that information saved in a document that you can update and use in the future.
One of the most intimidating aspects of moving to San Francisco was the fact that I’d have to rebuild my social life from the ground up. I’ve made a very concerted effort to get out of the apartment and explore the city, but exploring isn’t enough. I generally find it pretty easy to make friends, but it’s tough when you don’t have other friends to introduce you to their social circles. Meetup.com is a website where you can signup for “meetups” which are basically groups that schedule events open to the public. Think high school clubs for adults. There’s a meetup for nearly any interest you could have: meditation, yoga, photography, writing, professional networking, sports team fans, religion, live music…. and if the meetup you’re looking for doesn’t exist, go ahead and create it! It’s a great way to meet people with common interests and everyone who joins is usually very friendly because the whole point of it is to make new friends. Just be careful about your email communication settings. Joining a bunch of meetups can quickly flood your inbox, which can discourage you from seeing the value in the service and using it.
One of my favorite TV shows is Diners, Drive-ins and Dives on The Food Network. The food always looks incredible and the restaurants have a ton of character. ”Flavortown, USA” is one of the catch phrases from the host, Guy Fieri, as in “Wow, partner, that meatloaf sandwich is a trip to Flavortown, USA.” He has a ton of great catch phrases like that, but someone turned this one into a website that maps out all of the restaurants he’s featured in the show on a searchable google map. Each restaurant has it’s own page with details and most include the actual video clip from the show so you can see their signature dishes and what it’s like inside before you visit. This website is now one of the first places I go anytime I have a road trip planned and I’ve never been disappointed. In a new city, it can help you explore new neighborhoods and many locals usually know about the places, so it can often be a nice conversation starter.
This is a San Francisco-specific tool I’ve been able to use, but it’s been one of the most useful iPhone apps I’ve ever downloaded. It contains over 50 different apps, including tools to locate Wifi hotspots, tours, restaurants, bars, nightlife and guides to help you navigate public transportation, find local events and read local news among many other things. It’s a jack-of-all-trades app for living in San Francisco that would be helpful to anyone in the area, not just those of us who are new. If you aren’t moving to San Francisco, just try searching your phone’s App Store for your city’s name and see what comes up. You might be pleasantly surprised at what you find.
What did I miss?
Those were the most helpful tools I used during my relocation to San Francisco and I’ve been very pleased with just how smooth a transition it has been. I’ll keep adding websites, apps and other tools as I find them, but I’m curious to hear what other people have used. Do you have any suggestions for people moving to San Francisco or relocating in general? Please leave any feedback you have in the comments section below!
I’ve Joined Altimeter Group!
I am very excited to announce that I’ve joined the Altimeter Group! I’ve posted it already on Twitter, LinkedIn and Facebook, but I wanted to announce it here because I’m expecting to devote more time here as a result of this career move. I’m taking on the role of a researcher in support of the incredible analysts at Altimeter and will be spending most of my time working closely with Charlene Li (twitter, blog), Alan Webber (twitter, blog), and Jeremiah Owyang (twitter, blog).
Much of my research at Interpret LLC focused on the new technology disrupting the entertainment industry, but I’m eager to expand that into studying how disruptive technology (and especially social media) are affecting the best business practices in other industries. The level of research into this area at Altimeter Group is unparalleled so I’m both excited and humbled to join their excellent team at this time.
Of course this means a big move for me personally as I just relocated from LA to San Francisco, so keep an eye out for a post on all the websites and digital tools I used to aid my relocation.
The Digital Analyst
Zak Kirchner
