|Reviews a waste of time||downunderracer|
Jun 10, 2002 6:53 PM
|Sorry if this has been discussed before, but I'm finding more and more than the so-called reviews on this site are a waste of time. Most of the newer ones appear to be thinly disguised advertisements. Looks like the manufacturers are engaging in guerilla marketing campaigns and we are the losers. As an example, have a look under Trainers. The first one on the list, can't remember its name, has 20 reviews all giving it 5 chillies. And none of them list any weaknesses. Objective reviews - yeah right! And there's an ad for the same trainer right there on the page. Give me a break! |
RBR, how about having a review service with real reviews and where we can comment on any product, not just the ones you get paid to list?
|re: Reviews a waste of time||Jekyll|
Jun 10, 2002 7:11 PM
|Maybe the trainer marketing people are doing what Airborne used to do (maybe still does) - write a review get a free water bottle....
Everyone thinks that their bike or part, etc is the best. Most have not ridden another bike in a decade: Thus every review is positive. I try to look for negative reviews. If there is a consistent pattern of failures in the reviews for a particular product then there might be something to it. And if you think the reviews here are bad, try their car site.
Jun 10, 2002 9:56 PM
|One thing I wish everyone would do is, well, review stuff in the first place, as there are not many to begin with, but to also review everything they demoed along the way in choosing their stuff. I think this would especially help frame and fork comparisons for potential buyers, as people would have to actually think about pros and cons before writing an unconditionally positive review. Of course, everyone that didn't write an unconditionally positive review would have to deal with crap from other angry reviewers that don't appreciate someone dissing their pride and joy. How dare they say nonpositive things about something they don't actually own! True point though, as part of a fair review process should critique the product's longevity, and test rides don't cut it here. But as you note, a predominance of long-term reviews skews things further when most haven't ridden anything else in a good while. Which is why I think test rides should be respected as well, albeit in the appropriate context.
I don't know, you just have to spend time learning how to read THROUGH what people say to figure out what's bullshit and what isn't. Kind of like work. Ugh.
Jun 10, 2002 10:22 PM
|Doesnt mean it has to be a good review.
Your implication doesn't really make sense to me. Another thing to note on big expensive stuff like frames, I really doubt anyone is going to bash the product (bike, frame or the also mentioned car) that they have doubtless spent much time researching and much $ purchasing. If they felt the product was a pile of crap they probably wouldnt buy it would they?
That, I guess, is the problem with the review thing. People dont buy stuff that sucks. So maybe thats the answer too, rate stuff you dont buy, provide feedback as to why you didnt.
I can't see the status of the reviews changing with out the review writers. I'll do my best though, to give thoughtful and informative reviews on any piece I think I have enough experience with to talk about.
Jun 11, 2002 5:55 AM
|To expand a bit: Most people don't buy things, they are sold things. Spent many years in retail/wholesale world to see this pattern. There is also strong need for validation after the purchase, especially one that usually involves a significant investment.
I think in many ways people buy bikes because they appeal aesthetically or emotionally rather then due to their actual physical attributes. Most never really test ride anything else and when they do it is rarely in a controlled environment (different wheels, tires, saddles, etc). The rare soul that gets beat up at the forums for a "which bike to buy" post probably has spent a lot more time thinking about their purchase then the vast majority of reviewers.
This is why I am so "fond" of the negative reviews. If there are a number of reviewers with the same problem then there is probably enough information to merit seeing a pattern and a problem.
In some ways the reviews at MTBR are "better" then here. The crowd tends to be far more blunt and more apt to switch components at a rather regular clip. If you can plow through the plethora of juvenile rantings you can score a better glimpse of product performance. There seems far more willingness to call a spade a spade over there. Here, at times, its like a MB club meeting, "do you love your Benz Cliff?" "Why, of course Cedric, it is a Benz is it not?"
I do believe that when manufacturers get involved in motivating their customers to review products for third parties it skews the results. I guess all of this is a double edged sword. I would loath to see the reviews go but at times I loath reading them.
|You called that one right||pmf1|
Jun 11, 2002 4:59 AM
|Every damn bike in there gets 4.5 out of 5 stars. Most people have no point of reference having only ridden one bike (theirs). Very little information in there. |
Cars. Its always amused me that the JD Powers survey gets cited so much. Its conducted within 60 days of your buying the car. Of course you're going to like it, you bought it didn't you? And what is going to go wrong in 60 days? Talk about worthless.
Jun 11, 2002 4:08 AM
|If you go to mtbr.com and look at the reviews for said trainer, some of the names are the same and or the reviews are very close. Said company is either padding their reviews, or the people reviewing the product have too much time on their hands and post the almost identical review on both sites.|
Jun 11, 2002 5:07 AM
|Buyer Beware! I just try to use common sense when reading the reviews. As others have said, it's unusual to do anything but love your bike, even if it's not a Pinarello Prince or Colnago C-40. Everyone loves his or her bike, that's why they're here at RBR.
I find the reviews on consumables much more useful. For instance, tires, or bar tape, etc. A person goes through a couple of sets of tires a year so gets a good feel for comparative quality.
If you think the reviews are skewed, just think how difficult it is to go from shop to shop to shop trying to get honest, informed opinions on things.
|Have to be taken with a grain of salt...||biknben|
Jun 11, 2002 5:14 AM
|Many of the reviews are extremely positive or extremely negative. Most people don't bother to add a review unless they are passionate about what they want to say. The product is so good it changed their life or is sucked so bad it was a nightmare.
Like someone else said, look for patterns. If a few reviews site a similar problem then there's a good bet there may be an issue. Also, bad reviews seem to be more usefull.
I remember I read the reviews about a stem I was considering. A bunch of reviews told horror stories about stripping bolts and whatnot. I bought it anyway and paid close attention to the torque on the bolts. In the end, my bars were moving and the bolts stripped anyway. The reviews were right on as far as I was concerned.
|bad reviews disappear!||Alexx|
Jun 11, 2002 7:26 AM
|I know, because I once wrote a scathing review on a certain brand of tire that had let me down too many times. I did refer to this tire as a piece of $hite, so maybe tha's why it disappeared after a couple of days. Still, you never see those look-alike positive reviews (a.k.a. Airbourne Advert Specials) get pulled.|
Jun 11, 2002 7:48 AM
|Even more than negative reviews, I look for "weaknesses" in positive reviews. At least those will usually be honest... the content of the reviews, not the number of stars/chilis, convinced me to get a CrossCheck (that, and the price). I'll post an honest (really!) review myself as soon as my knee has healed enough to put some serious miles on it.|
|re: Well Then...Contribute...||jrm|
Jun 11, 2002 7:54 AM
|And start listing some "real" reviews.|
|Official Response - "reviews NOT a waste" (long)||gregg|
Jun 11, 2002 8:09 AM
|Downunderracer, thanks for posting the good topic. I will try to answer as much here as I can.
In short, biknben is correct in saying that there are a lot of 5 star or 1 star ratings and not much in between. People either really love their product or really hate it. The best method is too read the reviews and digest them for yourself. Extract what you can from what is written, not necessarily just the ratings alone.
First off, I am just glad that we GET reviews written. Without them, we would have no RoadbikeREVIEW. Having said that, it is a difficult task to get people to write clear, concise reviews on a given product. We layout guidelines and examples, but in the end, it is up to the person submitting the review.
Now, in regards to the example you stated, Jekyll is correct when he said maybe they did an "Airborne" kind of promotion soliciting reviews. RoadbikeREVIEW allows manufacturers to tell their customers, "Hey, buy our product, use it, and then please write a review on RoadbikeREVIEW about it." What we do NOT allow is for manufacturers to say, "Go write a good review only" or "Please do not write anything negative". As long as the manufacturer doesn't try to sway their opinion in any particular direction, I see nothing wrong with this. This provides the manufacturer with desired feedback directly from customers and helps us build a better resource here. Yes, both Airborne and 1Up advertise with us, but no, that does not sway what we (specifically, what "I") allow on the site. I'm sure if you read the Airborne reviews, you will see reviews with criticisms as well as compliments.
I do check for manufacturers falsely "padding" their reviews. And yes, I have even caught some of them doing it. When it is determined that a review is false, the review is removed and the reviewer is not allowed to post any more reviews. Is it possible for a manufacturer to pad reviews and for me not to catch it? Yes. There is a lot of data on this site, and it is tough to find everything all the time. Often times, I am alerted to these situations by fellow roadbikereview and mtbreview users like you.
Personally, I find more value in "negative" reviews (as long as they are well written). Where else will I find real world use of a product and honest opinions on how it works? I can read plenty of the good things about products in the magazines or the ads, what I am really interested in when I buy a product is, what is wrong or potential wrong with this product, if any.
If you read the "Top Ten Reasons Reviews are Removed" it may help clarify things too:
10. All you wrote was "it sucks, don't buy it"
9. You used profanity
8. You insulted or attacked other reviewers
7. You don't actually own the product
6. You posted your review multiple times
5. You tried to sell something
4. You ranted about poor customer service rather than reviewing
the product. Comments about customer service may be included
in your review, but it should not be your whole review.
3. Your review didn't contain enough content. 50 words is the
minimum for a complete review.
2. You didn't substantiate your review (positive or negative)
1. You didn't leave a valid email address
We do NOT remove negative reviews solely because they are negative. If
you have something negative to say, you MUST substantiate it with as
much detail as possible. If a manufacturer genuinely wants to contact
you to rectify the situation, but cannot reach you because you did not
leave a VALID email address, then your review may be removed.
Conversely, if you rave about how great a product is but you do not
say why or do not include enough detail, your review may be removed.
We do remove good reviews as well as bad ones if they are not subs-
tantiated in your review.
Bottom Line: Your Review Must Be Useful and Informative!
Hopefully I have answered some of your concerns here. As always, feel free to email me if you would like to discuss more or post here.
-gregg, Site Manager RoadbikeREVIEW
p.s. Alexx, if you wrote "this tire as a piece of $hite" then I'm sure I removed it. However, if you rewrote your review without saying that (in that manner), and your review is informative and helpful, I will leave it up. And fyi, I DO pull good rated reviews as well as bad ones, if they don't follow the guidelines. Good rated reviews are usually pulled because they aren't long enough or just say "this product is great. go buy it."
|you got my support, gregg nm||DougSloan|
Jun 11, 2002 9:05 AM
|So Many People Just Don't Answer the Question||jose_Tex_mex|
Jun 11, 2002 11:21 AM
|For all those who do read the question and answer it - thanks. Your help is invaluable.
However, I am alarmed by the amount of people who provide feedback on either a totally unrelated topic or on different subject matter.
Doesn't anyone actually read the question?
Also, there's definitely a bias in response wrt reviews. Those people who bought the product are not likely to say - "Yeah, this sucks, I got taken." With that said, there's normally one voice in the crowd who sounds honest and is not trying to convince themselves they made a mistake.
Jun 11, 2002 11:58 AM
|The reviews I can't stand are from people who just bought something and must tell us about it. They usually go like this:
"I've only ridden this thing once, but it is the sweetest thing I've ever used. It's awesome. Everyone should buy one.... 5 chiles!"
Then, the same guy, two months later:
"This thing is awful. It didn't last five rides before it started going haywire. I took it back to the shop and they said this happens a lot. I called up the company and they said I couldn't return it...."
Maybe reviews would be more useful if a lot of them weren't initial reactions. I don't want to hear about your bike or component until you've put enough mileage on it to matter.
|just take them for what they are?||DougSloan|
Jun 11, 2002 12:17 PM
|Maybe look at the reviews not as the end all of product evaluation, but instead simply for what they are. Initial impressions of products can be important, as well as long term experience. If I get a saddle and it hurts my rear in 10 miles, I'm done with it, and might want to share that. It is what it is. Now if I've put 10,000 miles on a saddle and it was comfortable every one of those miles (which I have), that might be much more significant, but still the initial impression means something.
Some products lend themselves to early evaluation more than others. How long does it take to evaluate a seat post or a handlebar? Tires, on the other hand, might take a while, because durability is one of the important factors. Nonetheless, if you had a tire that blew out the sidewall the first time you pumped it up, not much of an opportunity for long term evaluation.
As long as people are honest in what they are doing, why not just read them for what they are?
I pretty much ignore the chilies and go to the guts of the objective evaluation, ignoring all the "this sucks" and "this rocks" comments. I want facts.
|I use the reviews||p chop|
Jun 11, 2002 6:57 PM
|Even the ones that are all good, I can can look and see what kind of rider thought so and why. This played a bid part in jumping on a 5-year-old Litespeed -- which turned out to be a great move.
The only weakness in the reviews is the lack of them for some products or categories.
That reminds me... I'm going to get other there and post a couple for some things that I've been waiting to review until I used them long enough...