I've actually thought an enormous deal about this, and sent a message bugging phanna about the direction of a Brawl matchup chart. Here's my thoughts:
First, data should only be collected from the character subforums by knowledgeable mains of the characters. This data will begin to solidify as matchup guides and rankings are added to character guides. The important thing is to not base most of the chart off personal opinions of any one person, be it yourself or random March '08 posters in your thread.
Second, all data should be collected only upon the consensus of BOTH character subforums. This should be obvious: if The Jigglypuff board insists that Jiggs counters Pikachu in Brawl, and the Pikachu board is saying that Pikachu counters Jigglypuff, part of this project would be working towards an agreement with the two bodies. In a case like this I would expect one of the advocating groups to merely be less informed than the other.
Third... dividing up characters.
-Zelda and Shiek are different characters. When tiers are calculated from the finished matchup chart, Both Shiek and Zelda's values should be slightly biased towards the other if it is higher. In rare cases where both the matchups are equal and an advantage is gained by actively switching between both (Shiek to build damage and Zelda to kill?), then for tier calculations the assignment may be inflated even higher.
(For example if we are using phanna's 0-10 system: Let's say that Shiek has an 8 against Mario while Zelda has a 2. Strictly for the purposes of calculating tiers (not on the chart itself), Zelda might be considered a 5 to acknowledge Shiek's available advantage. Second, let's say Zelda is considered a 5 against Pikachu, Shiek is also considered a 5 agaisnt Pikachu, and there is no advantage to be had by switching between the two. Both would be considered a 5 for purposes of tiers. Finally, let's say Shiek and Zelda also both have 6s against DK, but due to different reasons: perhaps Shiek can combo DK very well below 60%, and Zelda has relatively low-percent KOs on him. Not only this, but let's assume in our exmaple that Zelda/Shiek players find it realisticly possible to change characters without penalty due to DK's slow speed and recovery; both characters might be considered a 7 for purposes of tier calculation to reflect the advantage switching offers.)
-Samus and Zamus are also different characters. For tiers, a similar but smaller bias should exist for Samus, though obviously not Zamus.
(For example as above: Let's say Samus has a 3 against Mario while Zamus has a 7; we might consider Samus a 4 when calculating tiers. If the roles were reversed, Zamus would still be considered only a 3 however.)
-Pokemon Trainer deserves not only all three Pokemon on the chart, but an overall ranking too due to the unique mechanics of the character. Typically, we can assume a weight of two-thirds to the best matchup, one-third to the middle, and no weight to the worst. Pokemon Trainer would have his own spot on the tier list, and it would probably be artificially high since it would assume equal skill with all three characters. Individual Pokemon should also be put on the tier list according to their individual values, but penalized to reflect a stamina an switching penalty. (The cost of playing as one Pokemon exclusively.) Finally, like Zelda/Shiek, it is possible that a teamwork-bonus of sorts could apply in some situations.
(Couple examples: Let's assume the Pokmon have the following matchup against say DeDeDe: Squirtle 9, Ivysaur 6, Charizard 2. It can be fair to say that players will use Squirtle extensively, and occasionally Ivysaur while skipping Charizard as much as possible; PT would probably be given an overall rating of 8. Second example, perhaps Squirtle is a 3 against Marth, while Ivysaur and Charizard are both 6 for different reasons. PT's overall rating against Marth might be a 7, reflecting the combined advantages of Ivysaur and Charizard while largely ignoring Squirtle.)
I hope everything here makes sense.
