I’m going to use a scored system for evaluating linux distros for first-timers. It will be something like this:
=== proposed scorecard follows===
Suitability for folks migrating from XP.
Install: Is the installation procedure easy to understand and use with no unecessary jargon?
Default applications: What kind of browser is installed? Are there Office-replacement applications (either as a suite or standalone apps)?
Speed: How fast is this linux install on the kind of hardware that would have originally come with XP? This is important to squeeze the life out of that old hardware. To test this I run the distro as a virtual system; a kind of “computer inside a computer” the way your TV might do “picture in picture”. The virtual runs on meager (simulated) hardware: 512MB RAM, 12MB video RAM, 8GB hard drive, etc. Hopefully a good deal less than your XP machine has.
Updates: How are updates handled? How about version upgrades?
Popularity: this may seem silly, but remember that most linux support is user-to-user on forums, FAQs, websites, etc. There will be more users familiar with a popular version than with an obscure one. Same way that someone in the cafe might know about a Chevy but probably not a Lancia or Pugeot. Yes, the basics are the same but details vary.
Grandma test: Once installed, could you set your grandmother in front of it and have her be comfortable and productive on it? This test is named after a now-defunct podcast that used this term to ground their review in the real world of users.
Total: add’em up. Divide by the number of categories for a percentage rating.