My view is that the present procedure used by various digicam reviewers is lacking in objectivity and impartiality. I believe that personal bias plays a key part in most digicam reviews and results are often consistently slanted in favor of certain brands and against others. Reviewers may feel that they are providing useful, objective reports, but the opposite is usually the case in my opinion. The basic description of camera features and how they work may be satisfactory, but most reviewers cannot seem to avoid placing emphasis on features they think are important and minimizing the value of features they think are insignificant. Often, derogatory remarks are made about certain brands or features provided by those brands. This then colors the reviewers overall evaluation and conclusions concerning a particular product resulting in reports that are more subjective than objective

What I would like to see is a point scoring system similar to that which some auto testers and other evaluators of products use. The total number of points possible might be set at 100. Each feature of a camera could be assessed a maximum number of points in accordance with their perceived importance. For example, if pixels are considered the most important feature, pixels might be assigned a maximum rating of 8 points. Since 2-megapixel cameras are the max available right now, cameras with 2-megapixels would receive the full 8 points. Since 640 by 480 is basically the bottom end, those cameras would receive 0 points for pixels. A maximum of six points might be assessed for zooms such that the digicam with the most powerful zoom would receive the full 6 points and cameras with no zooms would receive 0 pointsothers in between would get scored accordingly. As new features became available, they would be assessed a maximum number of points and other features would have their maximum number of points reduced so that the total possible would always remain at 100.

The value of this system would be that points are given for all possible featuresand 0 points for cameras which dont have a particular featurescoring would be impartial across the board for all digicams. This would go a long way toward eliminating reviewer bias, especially as the point scoring system for many features would provide points that are not subject to opinion. For example, the zoom scoring (at this time) could be as follows (power divided by 14 times 6):

14X = 6 points 10X = 4.3 points (10/14 X 6) 5X = 2.1 points (5/14 X 6) 4X = 1.7 points (4/14 X 6) 3X = 1.3 points (3/14 X 6) 2X = .9 points (2/14 X 6) 0X = 0 points

Whenever possible, the points assessed would be determined in a mathematically supportable way such the personal opinion would not a factor. The maximum points possible for each digicam feature could be determined by surveying digicam owners and averaging the results. That is, owners could be asked to assess maximum points for each feature in accordance with their opinion as to the value of that feature, keeping the total to 100 points. Averaging the results and rounding to the nearest whole number could set the points for the system. From time-to-time, the survey could be repeated to correct for new features. This would provide a real-world value for each feature rather than a single-person value (as is done currently).

The most significant value of this system would be that, for the most part, scoring would be logical, repeatable, and unbiased. Judgement might enter into some areas such as ergonomics, quality, ease of use, convenience, versatility, etc. if such factors are made a part of the scoring system; however, this would still be far superior to the present process whereby recommendations are based entirely on the personal opinion of a single reviewer.

If enough owners and forum participants show support for such a system, one or more reviewing organizations might be persuaded to adopt practices similar to those above. This, in turn, would benefit all users of such reviews.

Rodger

-- Rodger Carter (rodger.carter@wpafb.af.mil), August 17, 1999

Answers

Hey...maybe if you post this on enough forums someone will listen :)

Anyway, I really don't think you can assign a cold mathimatical score to a digicam. There are so many intangibles when it comes to their use and operation. If we were testing PCs or say....microwaves, then the scoring system might work. PCs and microwaves perform very similar functions and their egronomic properties play a small part in their ultimate usefulness....but digicams? For me using a digicam is a very personal experience. It's very important for the camera to "feel right" If the camera dosen't feel right, then I wont be in the best frame of mind to take that wonderful photo in the first place. I own a Nikon 950. I like the way it feels. I like the way it operates. I don't get that same feeling with the olympus C2000. Im sure there are others who feel the opposite. If I went ahead and gotten an OLY, becuase it beat my 88.57 Nikon with its "superior" 91.15 score, I could imagine myself much less inspired. I think people are inteligent enought to exampine the specs of various camera, and add that to the much more important 'gut-feeling' from actually using the camera. If I would have blindly followed the suggestion of a ridgid scoring system, who knows how many of my favorite shots would have never seen the light of day. Also, certain PC magazines have tried scoring digicams lately using their own numerical score, and you know how valuable those reviews are! ;) Let's also not forget the invaluable addition of word-of-mouth. Do you actually believe all the people posting positive responces about the 950 are a case of mass hysteria? Is it any coincidence that it still very hard to locate a 950 in the retail market? I say use your own brain. Take it all in then get what is best for YOU.

-- andy (andycure@hotmail.com), August 17, 1999.

A scoring system would augment, not preclude, a tradition single-user review. Both together would provide more information than either alone. The proposed system does seem a little over-wrought to me, however. A simple benchmark score might be almost as useful and considerably easier to implement. Simply set a score (of ten, say) to each feature of some arbitrarily selected (or imagined) camera, and then scale scores up or down proportionally for other cameras. Readers could easily perform their own weighting based on personal preference / inten

-- Kirk Markley (nkm@phoenix.net), August 17, 1999.

My response was truncated. It should end "intended use."

I also neglected to mention that I did my own feature-benchmarking for a few cameras I was considering. Very helpful.

-- Kirk Markley (nkm@phoenix.net), August 17, 1999.

Recently megapixel.net started using a new numeric rating system. I have found their written reviews to be more informative and more useful when comparing cameras and their features. In determining how to weight individual features of the camera in a numerical rating system a bias is being set. One person may think resolution is the most important feature, the next battery life. With the written reviews you are given the oppritunity to agree or disagree with the writer emphasis of individual features. With a number that is lost since the biasing will be hidden in the math. Imaging-resource is natorious for down playing the negative side of the cameras they review, such as the slow update rate of kodak LCD viewfinders. It is up to us as readers to determine what the writer's style is. If imaging-resource want to be optimist and not emphasising the negative that is their right. It up to us to decide where we want the emphasis placed and judge for ourselves what is important.

I personally want long battery life in a compact camera with optical zoom. Is that your preference? Probably not, but looking at a number rating we probably wouldn't be able to tell which camera would fit each of our needs.

-- Bob G. (rgreg88721@hotmail.com), August 19, 1999.

Some sights already have a 1-10 rating system. I personally don't give these types of number ratings a great deal of weight.

Each individual user, reading reviews for particular cameras, has personal criteria to keep in mind when making a decision prior to purchasing a camera.

As an example: My funds are very limited and a USB interface and included battery charger are aspects of a camera that have great influence on my decision to purchase. Being a former film photographer and having worked as a professional for a number of years, I am aware of the benifits of auto exposure lock, spot and center weighted metering, lenses that are capable of attaching accessories too, and so forth. A good review will allow me to make an educated decisions regarding what features I can afford and allow me to arrive at a reasonable compromise of features I want and need and features I can afford.

With the preceeding paragraph in mind, I need more complete information regarding all functions of a camera. I like to see reviewers voice their dislikes and personal preferences. I don't like the number of reviewers who have commented on the 950's battery door popping open unexpectedly. I appreciate that sort of info. That sort of info (reviewers personal dislikes) helps me make a decision. Can I live with a Toy Company battery door? I'd like to have the info supplied so I can make the decision. If the zoom lens is noisy, I'd like to hear about it! Photographing a wild bird after a long stalk would be very frustrating when the noise from the zoom sends the creature to flight! If I'm photographing drag races then, a noisy zoom doesn't mean a thing to me! We don't need a number system. A number 10 camera to a group of reviewers may be a number 7 camera to me. We need complete, in depth reviews, so we as consumers can make the final decision.

-- Robert Patterson (rdavid@ix.netcom.com), August 19, 1999.

Um...

By the way Einstein...It's the CF door, not the battery door...and if you let a few reports of the CF door popping open (which goes away with a 10sec fix) then you don't DESERVE a 950!

Humph!

-- ferrell (ferrellcat@hotmail.com), August 28, 1999.

This site has been quite unbiased in my opinion. A bunch of us have been pleaing for MORE opinion in the reviews (as long as it is clearly seperated and noted as such). The reviews are really good, but I've noticed fewer and fewer negative comments or negative comparisons against other cams. Seems like one gush after another, well they are mostly really good products, but I'd like to see more negatives listed.

This may be unfair or untrue but my perception is that this site is increasingly hesitant to post negative comments about cameras they review, perhaps for fear of sabotaging their relationship with manufacturers who give them cams to review?

-- benoit (foo@bar.com), August 30, 1999.